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^^^^^^^^^Accountability  *  Integrity  *  Reliability _ 

United  States  General  Accounting  Office 
Washington,  DC  20548 


November  16,  2001 

The  Honorable  Henry  J.  Hyde 
Chairman 

Committee  on  International  Relations 
House  of  Representatives 

Dear  Mr.  Chairman: 

Dramatic  changes  continue  to  occur  in  the  world’s  economic,  political, 
technological,  and  environmental  landscapes.  Increases  in  global  trade, 
transnational  terrorism  and  organized  crime,  and  international  health 
concerns,  for  example,  are  combining  to  create  a  more  complex,  more 
vulnerable  world  scene.  To  promote  U.S.  interests  in  the  face  of  such  rapid 
change,  24  federal  agencies  are  collectively  engaged  in  foreign  affairs 
activities  at  255  overseas  locations  in  162  countries.^ 

As  our  nation’s  lead  federal  agency  for  foreign  affairs,  the  Department  of 
State  (State),  among  other  things,  is  responsible  for  coordinating  and 
supporting  federal  agencies’  international  activities,  such  as  providing  a 
means  for  effective  interagency  information  sharing.  Currently,  this  means 
does  not  exist,  resulting  in  agencies  not  getting  the  right  information  to  the 
right  people  at  the  right  time.  To  address  this  situation,  State  is  leading  a 
multiagency  program  to  modernize  the  information  technology  (IT) 
environment  supporting  federal  agencies’  overseas  operations.  In  this  role, 
State  is  acting  on  behalf  of  the  other  foreign  affairs  agencies  to  acquire  and 
test  a  common  knowledge  management  prototype  and  pilot  system.^ 


Vhese  agencies  include  the  Departments  of  Agriculture,  Commerce,  Defense,  Energy, 
Health  and  Human  Services,  Interior,  Justice,  Labor,  Transportation,  Treasury,  and 
Veterans  Affairs;  the  Environmental  Protection  Agency,  the  Federal  Emergency 
Management  Agency,  the  General  Services  Administration,  the  National  Aeronautics  and 
Space  Administration,  the  Library  of  Congress,  the  Social  Security  Administration,  the 
American  Battle  Monuments  Commission,  the  Agency  for  International  Development,  U.S. 
Trade  Representative,  the  Overseas  Private  Investment  Corporation,  the  Trade  and 
Development  Agency,  the  National  Science  Foundation,  and  the  Peace  Corps.  These 
agencies  are  represented  overseas  in  162  countries  at  255  posts  (162  embassies,  76 
consulates,  17  other  missions  and  offices). 

^Knowledge  management  involves  the  use  of  business  processes  and  intellectual  and 
technological  assets  to  promote  and  provide  for  collaboration  and  information  exchange 
for  the  purpose  of  accomplishing  mission  goals  and  objectives. 
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This  report  provides  our  assessment  of  State’s  efforts  to  lead  this  program 
to  modernize  the  IT  environment  supporting  federal  agencies’  overseas 
operations.  As  agreed  with  your  office,  our  objectives  were  to  determine 
(1)  what  State’s  plans  are  for  acquiring  a  multiagency,  common-platform 
knowledge  management  system  and  (2)  whether  State  has  effective 
controls  in  place  for  acquiring  this  system,  that  is,  defined  roles  for  State 
and  its  agency  partners^  and  effective  controls  over  its  investment, 
enterprise  architecture,  and  system  acquisition  management  processes.  In 
conducting  this  review,  we  compared  State’s  current  and  planned 
activities  against  legislative  requirements,  federal  guidance,  and  best 
practices  relevant  to  systems  modernization.  Our  objectives,  scope,  and 
methodology  are  presented  in  more  detail  in  appendix  I. 


Results  in  Brief 


State  is  in  the  early,  formative  phase  of  a  long-term  plan  to  acquire  and 
deploy  a  common  knowledge  management  system  for  overseas-based 
agencies  engaged  in  foreign  affairs  activities.  This  system  is  to  provide 
functionality  ranging  from  basic  Internet  access  and  e-mail  to  mission- 
critical  policy  formulation  and  crisis  management  support.  While  detailed 
long-term  modernization  plans  understandably  do  not  yet  exist,  near-term 
plans  do.  These  plans  show  that  State  will  focus  first  on  prototyping  and 
pilot  testing  a  system  to  better  understand  user  requirements  and 
alternative  design  options.  The  plans  also  show  that  State  has  assumed 
responsibility  and  accountability  for  funding  and  leading  the  management 
and  administration  of  these  near-term  activities.  System  prototyping  and 
pilot  testing,  which  involve  investing  a  relatively  small  amount  of  time  and 
resources  in  building  and  evaluating  a  much  simpler  version  of  the 
operational  system  solution,  can  be  effective  risk-reduction  measures  for 
large  system  modernization  programs.  State  currently  plans  to  complete 
prototype  evaluation  by  May  2002  and  pilot  testing  by  September  2002. 

To  lead  the  near-term  activities.  State  is  employing  informal  management 
controls,  which  are  adequate  given  the  department’s  stated  purpose  and 
scope  of  these  activities.  However,  acquiring  and  deploying  system 
capabilities  that  are  intended  for  operational  use,  particularly  a  system  like 
this  that  involves  multiple  agencies  and  performs  mission-critical 


^State’s  primary  agency  partners  include  the  U.S.  Agency  for  International  Development, 
the  Peace  Corps,  and  the  Departments  of  Defense,  Justice,  Treasury,  Agriculture, 
Transportation,  and  Commerce.  Together,  State  and  these  eight  agencies  represent  99 
percent  of  our  overseas  presence.  State  and  the  Department  of  Defense  represent  about  80 
percent  of  our  foreign  overseas  presence. 
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functions,  requires  a  much  greater  level  of  management  discipline  than 
that  needed  for  system  prototyping  and  pilot  testing.  Specifically,  the  more 
rigorous  management  controls  that  are  needed  include  (1)  explicitly 
defining  a  multiagency  governance  and  funding  structure  for  acquiring  and 
deploying  the  operational  system;  (2)  ensuring  that  time  and  resources  are 
invested  in  economically  justified,  architecturally  compliant  increments  of 
the  operational  system  and  verifying  that  increments  meet  return-on- 
investment  expectations  before  investing  heavily  in  later  system 
increments;  (3)  having  and  using  an  overseas-presence  enterprise 
architecture  as  a  blueprint  to  guide  and  constrain  system  investments;  and 
(4)  having  and  following  mature  and  disciplined  software  acquisition 
management  capabilities.  To  date,  it  is  appropriate  that  State  has  not  yet 
established  these  controls  because  its  focus  has  been  on  evaluating  system 
prototypes.  However,  without  these  more  rigorous  controls,  it  is  unlikely 
that  State  and  its  agency  partners  will  deliver  needed  operational  system 
capabilities  on  time  and  within  budget. 

During  the  latter  stages  of  our  review.  State  initiated  steps  to  establish 
some  of  these  management  controls  and  committed  to  establishing  others. 
However,  much  remains  to  be  accomplished,  and  plans  do  not  yet  exist  for 
establishing  some  controls.  Accordingly,  we  are  making  recommendations 
to  the  Secretary  of  State,  as  the  designated  lead  official  for  the  overseas- 
presence  system  modernization  program,  that  are  intended  to  ensure  that 
each  of  these  control  areas  is  fully  addressed  before  acquisition  of  the 
operational  system  solution  begins. 

In  commenting  on  a  draft  of  this  report.  State  agreed  with  our  findings  and 
conclusions  that  its  management  controls  over  near-term  prototype  and 
pilot-testing  activities  are  adequate,  but  disagreed  that  the  department’s 
controls  for  managing  the  next  phase  of  the  program  are  not  adequate. 
Additionally,  State  commented  that  our  recommendations  should  be 
deleted  because  (1)  we  reported  that  management  controls  over  near-term 
activities  are  adequate  and  thus  corrective  actions  are  not  needed;  (2) 
except  for  defining  agency  roles  and  responsibilities.  State’s  existing 
management  controls  are  adequate  for  the  next  phase  of  the  program;  and 
(3)  the  lead  agency  for  the  next  phase  of  the  program  is  uncertain  and  thus 
the  recommendations  are  premature. 

We  disagree  with  State’s  comments.  First,  the  level  of  management  rigor 
and  discipline  needed  to  effectively  manage  prototype  and  pilot-testing 
activities,  as  discussed  in  the  report,  is  not  as  demanding  as  that  needed 
for  acquisition  and  deployment  of  a  system  that  is  to  be  used 
operationally.  Thus,  for  example,  while  State’s  lack  of  an  enterprise 
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architecture  to  guide  and  constrain  prototype  and  pilot-testing  activities 
does  not  introduce  significant  program  risk,  it  would  introduce 
considerable  risk  for  State  to  begin  the  next  phase  of  the  program  without 
this  architecture.  Second,  our  recommendations  are  appropriate  and 
warranted  because  (1)  they  are  not  intended  to  correct  problems  with 
management  of  the  prototype  and  pilot  activities,  but  rather  to  proactively 
provide  a  recommended  road  map  for  how  to  successfully  manage  the 
next  phase  of  the  program;  (2)  the  existing  controls,  as  explained  above, 
are  not  adequate  for  the  next  phase  of  the  program;  and  (3)  even  though 
the  possibility  exists  that  State  might  not  be  the  lead  agency  for  the  next 
phase  of  the  program,  prudent  planning  dictates  that  the  agencies’  roles 
and  responsibilities  for  the  next  phase  be  clearly  defined  and  agreed  to 
sooner  rather  than  later  to  provide  continuity  to  the  program  and  avoid 
unnecessary  delays. 

State’s  written  comments,  along  with  our  responses,  are  reproduced  in 
appendix  11. 


Background 


In  response  to  the  tragic  1998  bombings  in  Kenya  and  Tanzania,  the  former 
Secretary  of  State  established  an  independent  panel,  the  Overseas 
Presence  Advisory  Panel  (Panel),  to  analyze  our  overseas  presence  and 
develop  recommendations  for  organizing  and  managing  our  embassies  and 
consulates  overseas.  After  visiting  several  overseas  posts,  the  Panel 
concluded  that  the  condition  of  U.S.  posts  and  missions  abroad  was 
unacceptable  and  recommended  eight  major  types  of  changes,  which, 
according  to  its  report,  would  improve  the  efficiency  and  effectiveness  of 
our  overseas  presence.''  Specifically,  the  panel  directed  the  President,  the 
Secretary  of  State,  and  congressional  leaders  to  form  a  partnership  to 
implement  the  following  eight  recommendations: 

improve  security  and  establish  accountability  for  security  at  our  overseas 
facilities; 


‘^America’s  Overseas  Presence  in  the  21st  Century:  The  Report  of  the  Overseas  Presence 
Advisory  Panel,  U.S.  Department  of  State  (November  1999).  Three  other  studies  of  the 
condition  of  our  overseas  posts  have  been  performed  during  the  last  3  years.  These  studies 
are  Road  Map  for  National  Security:  Imperative  for  Change,  The  United  States 
Commission  on  National  Security/21st  Century  (February  15,  2001);  Independent  Task 
Force  Report:  State  Department  Reform,  Council  on  Foreign  Relations  (February  2001); 
and  Equipped  for  the  Future:  Managing  U.S.  Foreign  Affairs  in  the  21st  Century,  The 
Henry  L.  Stimson  Center  (October  1998).  All  these  studies  cited  the  need  to  upgrade 
information  technology  at  U.S.  overseas  locations. 
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•  create  the  right  size  and  shape  for  our  overseas  presence,  ensuring  that 
posts  are  not  over-  or  under-staffed; 

•  ensure  that  U.S.  facilities  are  quickly  and  efficiently  renovated  and  are 
properly  maintained  and  managed  on  an  ongoing  basis; 

•  adopt  best  practices  for  human  resources  management,  such  as  improving 
recruitment  and  expanding  training  and  promotion  opportunities; 

•  invest  in  up-to-date  information  technology,  including  providing  Internet 
access  and  e-mail  to  all  staff; 

•  improve  and  reinforce  consular  services,  such  as  same-day  processing  of 
visas; 

•  enact  cost-  and  time-saving  administrative  reforms,  such  as  consolidating 
some  functions  (e.g.,  accounting  practices)  and  automating  others;  and 

•  reinforce  the  responsibilities  and  authorities  of  Ambassadors,  who 
represent  the  interests  of  the  President  of  the  United  States. 

Because  State  is  the  lead  agency  for  foreign  affairs,*^  President  Clinton 
directed  the  former  Secretary  of  State,  on  February  10,  2000,  to  lead  a 
cabinet-level  committee  to  implement  the  Panel’s  recommendations.  As 
part  of  the  former  Secretary’s  response,  the  Under  Secretary  for 
Management  formed  an  interagency  Overseas  Presence  Committee  (the 
Committee)  in  February  2000  to  address  three  of  the  eight 
recommendations.  The  Committee  then  established  three  subcommittees 
to  implement  the  Panel’s  recommendations:  Overseas  Facilities, 
Interagency  Rightsizing,  and  Interagency  Technology.®  (See  fig.  1  for  the 
Committee’s  structure  and  table  1  for  allocation  of  responsibilities.) 


®As  the  lead  agency  representing  U.S.  interests  overseas,  State  is  vested  with  a  wide  range 
of  responsibilities,  including  formulating  U.S.  policy  on  diverse  international  issues, 
coordinating  and  implementing  U.S.  government  programs  and  activities  overseas,  and 
influencing  other  countries  to  adopt  policies  and  practices  consistent  with  U.S.  interests. 

^According  to  State’s  Director  of  the  Office  of  Management,  Policy,  and  Planning,  this 
organizational  arrangement  is  currently  under  review  for  potential  restructuring  at  a  later 
date. 
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Figure  1 :  Management  Structure  Established  to  Address  Several  Panel  Recommendations 


"FASI  =  Foreign  Affairs  Systems  Integration. 
Source:  State. 
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Table  1:  Summary  of  Management  Responsibilities  and  Functions  of  the  Overseas  Presence  Committee  and  its  Entities 


Entity 

Responsibility/Function 

Overseas  Presence  Committee 

Established  to  implement  the  Panel’s  recommendations  (in  its  1999  report)  regarding 
overseas  facilities,  interagency  rightsizing,  and  information  technology.  Chaired  by  State’s 
Under  Secretary  for  Management. 

Overseas  Facilities  Subcommittee 

Tasked  with  seeking  new  ways  to  manage  and  finance  overseas  facilities,  such  as 
evaluating  the  option  of  charging  rent  to  the  occupants  of  overseas  diplomatic  facilities. 
Chaired  by  the  Deputy  Assistant  to  the  Secretary  of  State. 

Interagency  Rightsizing  Subcommittee 

Directed  to  address  issues  such  as  the  size  and  shape  of  overseas  posts,  the  streamlining 
of  every  mission,  the  reallocation  of  all  personnel,  and  budget  savings  from  reducing  the 
size  of  overstaffed  posts.  Chaired  by  the  U.S.  Deputy  Representative  to  the  United 

Nations. 

Interagency  Technology  Subcommittee 

(iTsr 

Charged  with  defining  the  high-level  functional  requirements,  selecting  the  specific 
enabling  strategies,  and  identifying  the  needed  funding  for  the  common  overseas  system 
through  its  two  permanent  working  groups — knowledge  management  and  information 
technology.  Chaired  by  State’s  Chief  Information  Officer  (CIO)  and  includes  the  CIOs  of 
State’s  agency  partners  for  this  initiative. 

Knowledge  Management  Working  Group*’ 

Established  within  the  ITS  to  facilitate  information  sharing  to  support  a  collaborative  work 
environment.  Charged  with  defining  the  operational  requirements  for  knowledge 
management,  recommending  enabling  strategies,  identifying  resources  to  implement  those 
strategies,  providing  oversight  and  guidance,  and  identifying  needed  funding.  Led  by 

State’s  Deputy  CIO,  who  is  also  State’s  Chief  Knowledge  Officer,  and  inciudes 
representatives  from  State  and  its  agency  partners. 

Information  Technology  Working  Group*’ 

Established  within  the  ITS  to  provide  the  foreign  affairs  community  with  a  common, 
interoperable  technology  and  infrastructure.  Intended  to  identify  commercial-  and 
government-off-the-shelf  applications  and  Internet  and  Internet-like  technologies  to  support 
interagency  collaboration.  This  effort  is  to  include  analyzing  requirements  for  a  common 
overseas  platform  as  well  as  related  planning  tasks  such  as  design,  architecture,  and 
acquisition.  Led  by  State’s  Deputy  ClO/Chief  Knowledge  Officer  and  comprises 
representatives  from  State  and  its  agency  partners. 

Interagency  Capital  Financing  Working 
Group' 

Established  within  the  ITS  to  develop  a  capital  financing  strategy  for  funding  the 
implementation  of  the  Panel’s  IT  recommendations.  Led  by  State’s  Deputy  ClO/Chief 
Knowledge  Officer  and  comprises  representatives  from  State  and  six  of  its  eight  agency 
partners. 

FASI  Architectural  Working  Group 

Established  within  the  ITS  to  develop  and  implement  the  enterprise  architecture  for  the 
common  overseas  system,  ensuring  compliance  with  State  and  its  agency  partners’ 
existing  enterprise  architectures.  Led  by  State’s  Director  for  Standards,  Policy,  Planning, 
and  Architecture. 

FASI  Program  Office 

Created  to  plan  and  implement  the  common  overseas  platform.  Using  the  user  and  system 
requirements  devised  and  prioritized  by  the  Knowledge  Management  and  Information 
Technology  working  groups,  this  team  is  to  design  and  deploy  an  unclassified, 
interoperable  information  technology  infrastructure  and  knowledge  management  system 
that  is  intended  to  address  the  recommendations  in  the  Panel’s  report.  Led  by  State’s 
Deputy  ClO/Chief  Knowledge  Officer. 

"The  efforts  of  this  subcommittee  are  the  primary  focus  of  this  report. 


These  working  groups  were  combined  in  Aprii  2001 . 

'According  to  State,  membership  for  this  working  group  currentiy  consists  of  representatives  from  the 
U.S.  Agency  for  internationai  Deveiopment,  the  Peace  Corps,  and  the  Departments  of  Defense, 
Treasury,  Justice,  and  Agricuiture. 
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In  October  2000,  the  then  Secretary  of  State  reported  on  the  progress 
made  in  addressing  the  Panel’s  recommendations  for  seven  of  the  eight 
recommendations/  In  response  to  the  recommendation  to  invest  in  up-to- 
date  information  technology,  State  reported  that  it  had 

•  established  an  acquisition  and  deployment  strategy,  preliminary  system 
requirements,  a  high-level  implementation  plan,  and  an  architectural 
concept  for  a  common  overseas  knowledge  management  system 
infrastructure; 

•  established  September  2000  as  the  goal  for  completing  a  formal  program 
management  plan,  with  a  goal  of  revising  the  plan  by  April  2001  after 
completing  evaluation  of  a  prototype  system;  and 

•  established  December  2001  as  the  goal  for  completing  system  pilot  testing 
at  two  posts. 

On  August  8,  2001,  State  issued  its  final  status  report  on  its  efforts  to 
address  the  Panel’s  recommended  changes  to  the  Chairman  of  the  Senate 
Committee  on  Foreign  Relations.  With  regard  to  the  recommendation  to 
upgrade  information  technology.  State  reported  that  the  interagency  group 
has  developed  and  is  testing  a  prototype  of  the  common  overseas  system 
and  that  a  field  test  of  the  system’s  infrastructure  is  being  conducted  at 
pilot  posts  in  Mexico  and  India.  In  addition.  State  officials  told  us  that  the 
date  for  completing  system  prototyping  has  been  delayed  by  13  months  to 
May  2002,  and  the  date  for  completing  pilot  testing  at  two  posts  has  been 
delayed  by  9  months  to  September  2002.  Also,  the  three  vendors  selected 
to  develop  system  prototypes  are  not  scheduled  to  deliver  these  systems 
to  State  for  evaluation  until  November  2001. 


Common  Overseas  System 
Modernization:  A  Brief 
Description 


The  modernization  program  is  intended  to  put  in  place  a  common  overseas 
knowledge  management  system  to  facilitate  unclassified 
information/knowledge  sharing  among  foreign  affairs  agencies.  Key 
system  capabilities  are  to  include,  among  other  things,  interagency  e-mail, 
post-specific  news  and  information  web-based  links,  crisis  coordination 
support,  policy  formulation  support,  and  various  administrative  functions 
(e.g.,  accounting,  contract  management,  training,  and  travel).  According  to 
State’s  draft  program  management  plan,  the  department  will  acquire  and 
deploy  system  functionality  in  each  of  these  areas  in  three  increments,  or 
three  prioritized  groupings  of  user  requirements,  which  State  calls  pilot. 


^The  report  did  not  address  the  Panel’s  recommendations  for  enacting  cost-  and  time¬ 
saving  administrative  reforms. 
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full-scale  implementation,  and  deferred.  (See  table  2  for  planned  system 
functionality  by  increment.) 


Table  2:  Planned  System  Functionality  by  Increment 


Increment  1:  Pilot 

Type:  General  user  requirements 

-accommodate  Sensitive  But  Unclassified  and  Unclassified  information  and 
processing 

-ensure  access  to  system  during  working  hours  at  all  locations 
-support  moblle/traveling  users 
-provide  a  powerful  “smart”  search  capability 
-support  the  ability  to  identify  experts  in  foreign  policy,  technical,  and 
administrative  disciplines  (i.e.,  a  knowledge  management  locator  service) 

-enable  “self  service”  processing  (e.g.,  the  ability  for  users  to  update  records  and 

_ initiate  transactions) _ 

Type:  Interagency  e-mail 

-provide  antivirus  processing 

-enable  delivery  of  e-mail  to  the  user’s  existing  agency  e-mail  system 
-provide  access  to  e-mall  addresses  and  other  basic  information  on  participants  in 
the  Overseas  Presence  Advisory  Panel  system 
-enable  end  users  to  create  customized  directories  and  group  addresses 
-provide  e-mall  capability  between  post  and  non-U. S.  government  entities 

-support  private  and  public  mailing  lists  and  list  management  services _ 

Type:  News  and  information:  provide  access  to 

-administrative  information  (e.g.,  financial  bulletins,  per-diem  rates,  travel 
regulations) 

-post  or  regional  newsletters  and  bulletin  boards 
-“news  gathering  and  analysis”  activities 
-news  “clipping”  services 

-issue-oriented  databases _ 

Type:  Crisis  coordination:  provide  support  for 

-identifying  and  locating  employees  at  a  given  post 

-collaborating  and  conferencing  to  support  work  groups  and  information  sharing 

-coordinating  activities  (e.g.,  transportation,  medical  treatment) _ 

Type:  Policy  formulation:  provide  support  for 

-researching  issues  related  to  a  given  policy 
-publishing  of  current  policies  and  decisions 

-collaboratively  developing  policy  documents,  guidelines,  processes,  and 
decisions 

-identifying  responsible  departments  and  persons  for  a  given  policy  area _ 

Type:  Administration:  provide  support  for 

-international  Cooperative  Administrative  Support  Services  System 
-contacting  management  through  a  world-wide,  on-line  staff  directory 
-coordinating  and  managing  calendars  and  schedules _ 
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Increment  2:  Full-scale  implementation 

Type:  General  user  requirements 

_ -provide  support  for  teams  with  non-U. S.  government  members _ 

Type:  Interagency  e-mail:  provide 

-e-mail  forwarding  (inside  or  outside  the  collaboration  zone) 

-integration  with  workflow  tool(s) 

_ -embedded  URLs  (Internet  and  Intranet) _ 

Type:  News  and  information:  provide  access  to 
-local/regional  translation  services 
-specialized  subscription  services 
-commercial  telephone  directories 
-geographic  databases 
-mission  performance  plan  data 

_ -telegrams  (current  and  archival) _ 

Type:  Crisis  coordination 

-provide  support  for  notifying  and  disseminating  information,  orders,  and 

_ instructions  to  government  and  U.S.  citizens _ 

Type:  Policy  formulation 

_ -provide  support  for  bureau  and  mission  performance  plans _ 

Type:  Administration:  provide  support  for 
-training  and  human  resources 
-procurement 

-travel  planning  and  management 

_ -budgeting  and  financial  management _ 

Increment  3:  Deferred 
Type:  General  user  requirements 

-support  access  from  electronic  organizers,  cellular  telephones,  personal  digital 
assistants,  etc. 

_ -provide  access  to  agency  (noncommon-overseas)  systems _ 

Type:  Administration:  provide  support  for 
-motor-pool  scheduling 
-inventory  and  equipment  maintenance 
-property  management 

Source:  State. 

The  system  is  to  use  commercially  available,  Internet-based  technology  to 
enable  users  to  access  the  system  from  any  location  using  their  respective 
agencies’  existing  networks  and  operating  platforms.  The  desktop  Internet 
browser  is  intended  to  be  the  connection  to  the  common  overseas  system 
and  its  knowledge  management  tools.  A  common  website  is  to  serve  as  the 
portal  to  the  customized  database  applications,  files,  and  communication 
tools  to  be  shared  across  the  foreign  affairs  community’s  diverse, 
distributed  environment.  (See  fig.  2  for  a  simplified  diagram  of  the 
system.) 
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Figure  2:  Simplified  Diagram  of  Target  Overseas  Knowledge  Management  System 
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In  1999,  the  Panel  estimated  the  cost  to  acquire  and  deploy  this 
unclassified  system  at  about  $200  million,  assuming  the  use  of 
commercially  available  technology.  About  a  year  later.  State  estimated  that 
the  investment  in  the  system  through  fiscal  year  2004  would  be  about  $271 
million.  State  plans  to  update  this  cost  estimate.  According  to  State’s  draft 
program  management  plan,  the  estimated  cost  to  perform  system 
prototyping  and  pilot  testing  is  $18.2  million. 

The  conference  report  accompanying  the  Departments  of  Commerce, 
Justice,  and  State,  the  Judiciary,  and  Related  Agencies  Appropriations  Act, 
2001,  stated  the  conference  committee’s  expectation  that  State  would 
spend  $17  million  to  carry  the  program  through  pilot  testing.* *^  State  has  met 
with  subcommittee  staff  to  keep  them  informed  of  its  plans  and  progress, 
and  State  has  agreed  to  submit  a  spending  plan  for  the  pilot  project,  as 
directed  by  the  corrference  report.  Prototyping  activities  are  currently 
scheduled  for  completion  in  May  2002. 


Recent  Review  of  Common 
Overseas  System 
Modernization  Program 
Noted  Management 
Challenges 


In  Jime  2000,  we  testified  that  State  was  in  the  early  stages  of  planning  for 
the  overseas-presence  system  modernization — establishing  preliminary 
milestones,  developing  rough  cost  estimates,  and  formulating  a  program 
management  plan.'’  At  that  time,  we  expressed  concern  regarding  (1) 

State’s  ability  to  obtain  agreement  internally  as  well  as  from  the  foreign 
affairs  partner  agencies  in  devising  and  deploying  a  common  technology 
solution  and  (2)  State’s  lack  of  an  approved  architecture  to  guide  and 
constrain  its  own  IT  investments.  In  our  testimony,  we  stated,  among  other 
things,  that  it  was  important  for  State  to  (1)  carefully  plan  the  initiative 
estabhshing  realistic  goals  and  milestones,  (2)  install  the  needed 
management  and  oversight  accoimtability  to  properly  guide  the  acquisition 
of  the  system,  and  (3)  anticipate  the  steps  needed  to  gain  the  full 
cooperation  of  its  agency  partners. 


*H.R.  Conf.  Rep.  No.  106-1005,  at  294  (2000)  states,  in  part,  that  $17  million  shall  be  for  a 
pilot  project  to  establish  a  common  technology  platform  at  overseas  posts  pursuant  to  the 
recommendations  of  the  Overseas  Presence  Advisory  Panel.  The  conference  report  also 
directs  that  State  submit  a  spending  plan  for  this  pilot  project. 

^Foreign  Affairs:  Effort  to  Upgrade  Information  Technology  Overseas  Faces  Formidable 
Challenges  (GAO/T-AIMD/NSIAD-00-214,  June  22,  2000). 
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Office  of  Management  and  Budget  (0MB)  guidance  outlines  risk  reduction 
principles  that  are  consistent  with  best  commercial  practices  and,  if 
effectively  implemented,  can  increase  the  probability  that  system 
modernization  programs  will  succeed.^  Among  other  things,  this  guidance 
supports  (1)  prototyping  system  components  before  acquiring  or 
developing  the  operational  system,  (2)  using  commercially  available 
technology,  and  (3)  securing  system  stakeholders’  involvement  and 
agreement  on  products  such  as  management  plans  and  system 
requirements. 

State’s  near-term  plans  generally  provide  for  each  of  these  areas  of 
guidance.  In  particular.  State  is  currently  focused  on  acquiring  and 
evaluating  system  prototypes  that  are  to  employ  either  commercially 
available  and/or  government-used  hardware  and  software  products.  On 
July  5,  2001,  State  selected  three  vendors,  each  of  which  was  to  develop  a 
prototype  system  by  October  2001.  However,  according  to  a  State  official, 
because  of  security  and  infrastructure  concerns  identified  during  testing 
that  the  vendors  must  address,  the  systems  are  now  scheduled  for  delivery 
in  November  2001.  In  addition.  State  planned  to  evaluate  the  delivered 
prototype  systems  using  evaluation  criteria  that  were  to  be  developed  by  a 
private  firm  by  September  28,  2001.“  While  an  initial  draft  of  the  criteria 
was  provided  on  that  date,  according  to  a  State  official,  revisions  were 
needed,  and  the  deadline  for  finalizing  the  evaluation  criteria  was  changed 
to  mid-November  2001.  Although  these  events  will  delay  the  deadline  for 
completing  the  prototype  evaluation  one  month  to  May  2002,  piloting 
activities  will  not  be  affected  and  will  begin  as  scheduled.  Once  the 
prototype  system  solutions  have  been  evaluated.  State  and  its  agency 
partners  plan  to  use  the  best  capabilities  from  each  prototype  to  refine 
user  requirements  and  develop  a  specification  for  the  foundational  system 
capabilities  that  will  be  pilot  tested.  State  plans  to  pilot  test  these 
foundational  capabilities  in  India  and  Mexico  between  April  and 
September  2002.  According  to  State,  no  commitments  have  been  made  to 
proceed  beyond  this  pilot  test,  and  any  decisions  about  how  best  to 
proceed  will  have  to  involve  agency  partners,  0MB,  and  relevant 
congressional  committees. 


State’s  Near-Term 
Plans  Are  Focused  on 
Risk  Reduction 
Activities 


^°Office  of  Management  and  Budget  (OMB)  Circular  A-130,  Management  of  Federal 
Information  Resources  (November  30,  2000)  and  Capital  Programming  Guide,  Version 
1.0,  OMB  Circular  A-11,  part  3  (July  2000). 

“state  officials  told  us  that  the  deadline  for  the  criteria  was  changed  from  September  28, 
2001,  to  mid-November  2001  because  of  the  events  of  September  11,  2001. 
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In  addition  to  prototyping  the  system  and  using  commercially  available 
products,  State  has  also  taken  steps  to  engage  other  foreign  affairs 
agencies  in  its  prototyping  activities,  including  securing  stakeholder 
involvement  and  buy-in.  Specifically,  responses  from  the  seven 
modernization  program  partner  agencies  that  responded  to  our  survey 
showed  that  they  were  generally  satisfied  with  State’s  efforts  to  engage 
them  in  near-term  program  activities.  Additionally,  four  of  these  agencies 
stated  that  State  had  informally  obtained  their  concurrence  on  such  items 
as  program  plans  and  system  requirements.  As  for  the  other  three 
agencies,  one  stated  that  while  it  had  reviewed  some  of  the  program 
documentation,  it  had  not  participated  in  an  approval  process;  another 
stated  that  it  had  reviewed  program  documentation  for  informational 
purposes  only  and  was  unaware  of  any  approval  process;  and  the  third 
stated  that  it  had  not  reviewed  any  of  the  program  documentation. 

Nevertheless,  we  found  that  these  and  other  interagency  meetings,  such  as 
program  working  group  meetings,^  have  occurred  and  provided  forums 
for  informally  engaging  agency  partners.  In  general,  agency  partners 
attended  these  meetings  and  participated  in  the  program.  For  example, 
two  agency  partners  have  assigned  staff  to  the  FASI  Program  Office.  “ 

Available  documentation  and  State  CIO  officials’  statements  confirm  that 
agency  partners’  buy-in  to  program  decisions,  such  as  agreement  on  the 
requirements  for  the  prototype  and  approval  of  plans,  is  being 
accomplished  informally.  According  to  State,  representatives  for  the 
partner  agencies  agreed  that  their  lack  of  opposition  or  comments  equated 
to  concurrence  on  actions,  plans,  and/or  documents,  thereby  eliminating 
the  need  to  obtain  actual  signatures  to  indicate  approval.  Although  State 
could  not  provide  us  with  any  written  agreements  to  this  effect,  revised 
minutes  of  an  Interagency  Technology  Subcommittee  meeting  reflected 
this  informal  agreement. 


of  November  9,  2001,  the  U.S.  Agency  for  International  Development,  the  Peace  Corps, 
and  the  Departments  of  Agriculture,  Justice,  Transportation,  Commerce,  and  Treasury  had 
responded  to  our  survey;  the  Department  of  Defense  had  not. 

^^These  meetings  include  the  Knowledge  Management  Working  Group,  the  Information 
Technology  Working  Group,  the  Architecture  Working  Group,  and  the  Interagency  Capital 
Financing  Working  Group. 

^''Treasury  has  assigned  an  employee  to  State’s  Knowledge  Management  Team,  and  the 
Justice  Department  has  detailed  an  individual  to  assist  State  in  developing  the  overseas 
presence  enterprise  architecture. 
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Much  Remains  to  be 
Done  Before 
Acquiring  the 
Operational  System 


The  Clinger-Cohen  Act  of  1996,  in  combination  with  0MB  guidance 
implementing  the  act,  provide  federal  agencies  with  a  framework  for 
effectively  managing  modernization  efforts  that  are  consistent  with  best 
commercial  practices.  Among  other  things,  0MB  requires  (1)  a  clear 
definition  of  program  management  roles,  including  an  explicit  definition  of 
responsibilities  and  accountabilities  for  such  things  as  funding,  investment 
decisionmaking,  requirements  management,  architecture  definition  and 
integration  with  legacy  systems,  and  testing;  (2)  use  of  incremental 
investment  management  principles  to  minimize  the  enormous  risk 
inherent  in  large  modernization  programs  that  involve  many  things  to  be 
done  over  many  years;  (3)  the  development,  maintenance,  and 
implementation  of  an  enterprise  architecture,  which  is  an  institutional 
blueprint  for  guiding  operational  and  technological  modernization;  and  (4) 
the  use  of  effective  IT  project  management  processes,  such  as  processes 
for  managing  software-intensive  system  acquisition  projects. 

For  its  overseas  knowledge  management  system  modernization  program. 
State  program  officials  told  us  that  they  plan  to  establish  and  implement 
each  of  these  management  controls.  However,  much  remains  to  be 
accomplished  because  thus  far  State  has  devoted  its  attention  to  system 
prototyping  activities,  and,  according  to  State  officials,  they  first  had  to 
brief  congressional  staff  earlier  in  fiscal  year  2001  on  the  use  of 
appropriated  funds  before  establishing  these  management  controls.^® 

It  is  essential  that  these  controls  be  effectively  implemented  on  the 
overseas-presence  system  modernization  program  before  State  proceeds 
with  its  acquisition  of  the  operational  system.  To  do  less  would 
unnecessarily  increase  the  risk  that  needed  system  capabilities  will  not  be 
delivered  on  time  and  within  budget. 


^^Clinger-Cohen  Act  of  1996,  Public  Law  104-106  and  0MB  Circular  A-130,  Management  of 
Federal  Information  Resources  (November  30,  2000). 

^^Consistent  with  congressional  direction  to  submit  a  spending  plan  for  the  pilot  project. 
State  briefed  congressional  staff  on  its  plans  for  the  pilot  project  and  its  proposed  use  of 
the  funds  before  it  began  spending  the  $17  million. 
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Clear  Definition  of 
Stakeholders’ 
Accountability  Is  Critical 
to  Proceeding  Beyond 
Pilot  Testing 


OMB  guidance  recognizes  the  importance  of  fully  and  explicitly  defining 
and  ensuring  understanding  of  program  stakeholders’  areas  of 
responsibility  and  accountability  in  managing  and  funding  a  program. 
Similarly,  the  Software  Engineering  Institute  (SEl),  recognized  for  its 
expertise  in  managing  software-intensive  systems,  emphasizes  the 
importance  of  clearly  designating  roles  and  responsibilities  on  system 
acquisition  programs.  While  doing  so  is  essential  on  any  program,  it  is 
particularly  critical  for  programs  like  the  overseas-presence  knowledge 
management  modernization  program  because  its  success  depends  on  the 
effective  interplay  of  multiple  federal  agencies.  Without  well-defined, 
understood,  and  agreed-to  roles  in  such  areas  as  funding  and  acquisition 
management  (e.g.,  investment  decisionmaking,  requirements  management, 
architecture  definition  and  integration  with  legacy  systems,  and  testing). 
State  cannot,  for  example,  be  assured  that  necessary  resources  will  be 
provided  (e.g.,  financial,  human  capital,  and  technological)  and  critical 
tasks  executed. 

According  to  State,  the  roles,  responsibilities,  and  accountabilities  of 
foreign  affairs  agencies  have  been  formally  defined.  However,  State  did 
not  provide  us  with  documentation  to  support  this  position,  as  we 
requested.  Furthermore,  agency  partners  that  responded  to  our  survey 
stated  that  their  respective  roles,  responsibilities,  and  accountabilities  for 
this  initiative  have  not  yet  been  formally  defined. 

According  to  State,  as  the  principal  foreign  affairs  agency,  its  role  is  to 
provide  administrative  and  management  leadership  and  support  for  near- 
term  prototype  and  pilot-testing  activities,  and  it  is  the  agent  for  bringing 
the  foreign  affairs  agencies  together.  However,  State  said  that  it  is  not 
responsible  for  engaging  agency  partners  as  part  of  the  near-term 
prototype  and  pilot-testing  activities. 

As  noted  earlier,  given  the  stated  purpose  and  scope  of  these  near-term 
activities  and  State’s  leadership  role,  this  definition  of  roles  is  not 
currently  a  significant  issue.  However,  it  is  essential  that  State  and  its 
agency  partners  fully  define  agreed-to  roles,  responsibilities,  and 
accountabilities  for  program  funding  and  management  before  moving 
beyond  planned  near-term  activities.  To  do  less  would  unnecessarily  put 
the  modernization  program  at  risk. 
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Incremental  Investment 
Management  Could 
Reduce  Overall  Program 
Risks 


Incremental  investment  management  involves  three  fundamental 
components:  (1)  developing/acquiring  a  large  system  in  a  series  of  smaller 
projects  or  system  increments;  (2)  individually  justifying  investment  in 
each  increment  on  the  basis  of  costs,  benefits,  and  risks;  and  (3) 
monitoring  actual  benefits  achieved  and  costs  incurred  on  ongoing 
increments  and  modifying  subsequent  increments/investments  to  reflect 
lessons  learned.  Using  this  approach,  agencies  can  avoid  discovering  too 
late  that  their  systems  are  not  cost  beneficial  and  reduce  the  risks 
associated  with  large,  expensive  projects.  This  approach  allows  overlap 
and  smooth  transition  among  increments  because  lessons  learned  from 
actual  results  of  ongoing  increments  would  be  monitored  and  evaluated 
continuously  so  that  these  results  will  be  available  for  use  in  defining  and 
justifying  future  increments. 

The  Clinger-Cohen  Act  of  1996  requires  agencies  to  follow,  to  the 
maximum  extent  practical,  an  incremental  approach  to  investing  in  IT 
projects.  Also,  0MB  policy  requires  that  investments  in  major  systems  be 
implemented  in  increments,  with  each  increment  delivering  measurable 
benefits.^^  More  specifically,  OMB’s  Capital  Programming  Guide'®  describes 
the  use  of  modular  contracting,  or  incremental  investment,  including  its 
application  and  benefits.  In  particular,  0MB  states  that  project  increments 
should  provide  for  the  following: 

•  Separability:  Each  increment  should  be  fully  funded,  have  substantial 
programmatic  use  that  is  not  dependent  on  any  subsequent  increment,  and 
be  capable  of  performing  its  principal  functions,  even  if  no  subsequent 
increments  are  acquired. 

•  Interoperability:  Each  increment  should  comply  with  a  common 
architecture  or  commercially  acceptable  technology  standards  and  should 
be  compatible  with  and  capable  of  being  integrated  with  other  increments. 

•  Performance  requirements:  Each  increment’s  performance  requirement 
should  be  consistent  with  the  performance  requirements  of  the  completed 
overall  system  and  should  address  interface  requirements  with  other 
increments. 


"^OMB  Circular  A-130  (November  30,  2000);  Capital  Programming  Guide,  Version  1.0, 
OMB  Circular  A-11,  part  3  (July  2000),  pp.  546-572. 

Capital  Programming  Guide,  Version  1.0,  OMB  Circular  A-11,  part  3,  Supplement, 
Planning,  Budgeting,  and  Acquisition  of  Capital  Assets  (July  1997),  pp.  35-37. 
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In  short,  incremental  investment  helps  to  mitigate  the  risks  inherent  in 
acquiring/developing  large  systems  by  dividing  a  single,  large  program  into 
smaller,  independently  useful  components  with  known  and  defined 
relationships  and  dependencies.  It  is  well  understood  that  making 
investment  decisions  up  front  on  large-scale,  long-term  programs  is  risky 
because  their  economic  justification  is  based  on  costs,  benefits,  and  risks 
that  are  difficult  to  forecast  reliably,  partially  because  later  increments  are 
not  well  understood  or  defined  and  partially  because  they  are  subject  to 
change  in  light  of  experience  on  nearer  term  increments  and  changing 
business  needs.  Through  incremental  investment  management, 
organizations  can 

•  reduce  the  level  of  program  risk  and  complexity  faced  at  any  one  time  by 
spreading  this  risk  and  complexity  across  a  series  of  smaller  investments; 

•  permit  the  delivery  of  some  part  of  the  expected  business  value  earlier 
rather  than  waiting  until  later  for  the  total,  but  more  uncertain,  business 
value; 

•  continuously  monitor  and  evaluate  the  delivery  of  cost  and  benefit 
expectations  on  ongoing  increments  and  use  this  information  to  better 
define  and  economically  justify  these  increments;  and 

•  permit  later  increments  to  exploit  technology  advances  or  accommodate 
evolving  business  needs. 

According  to  modernization  officials,  preliminary  plans  provide  for 
acquiring  and  investing  in  the  overseas  knowledge  management  system  in 
three  increments:  pilot,  full-scale  implementation,  and  deferred.  However, 
State  has  not  yet  established  specific  plans  and  management  processes  for 
acquiring  the  operational  system  because,  according  to  State’s  CIO,  it  does 
not  have  congressional  authorization  to  do  so.  We  were  unable  to 
corroborate  State’s  position  as  to  why  it  has  not  established  these  plans 
and  processes. 

In  addition,  modernization  officials  stated  that  the  ongoing  system 
prototyping  and  planned  pilot-testing  activities  were  not  economically 
justified  as  a  separate  increment.  However,  these  officials  noted  that  all 
future  incremental  investments  in  the  system  will  be  economically 
justified,  and  the  results  and  lessons  learned  from  prior  increments  will  be 
used  to  define  and  make  investment  decisions  about  future  increment(s). 
Additionally,  they  stated  that  future  increments  will  also  be  evaluated  to 
ensure  that  retum-on-investment  commitments  are  met. 

It  is  critical  for  State  to  follow  through  on  its  stated  commitments  to 
implement  incremental  investment  practices.  Otherwise,  it  risks  making  a 
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single,  monolithic  investment  decision  for  the  entire  program.  This  “all  or 
nothing”  approach  to  investing  in  IT  has  historically  resulted  in  agencies 
investing  huge  sums  of  money  in  systems  that  do  not  provide 
commensurate  benefits,  and  thus  has  been  abandoned  by  successful 
organizations.  The  need  to  avoid  this  pitfall  was  a  major  impetus  for  the 
investment  management  reforms  of  the  Clinger-Cohen  Act. 


Enterprise  Architectures 
Are  Essential  Tools  for 
Guiding  and  Constraining 
Investment  Decisions 


Enterprise  architectures  are  essential  tools  for  effectively  and  efficiently 
reengineering  business  processes  and  for  implementing  and  evolving  their 
supporting  systems.  Enterprise  architectures  systematically  capture — in 
useful  models,  diagrams,  and  narrative — the  full  breadth  and  depth  of  the 
mission-based  mode  of  operations  for  a  given  enterprise,  which  can  be  (1) 
a  single  organization  or  (2)  a  functional  or  mission  area  that  transcends 
more  than  one  organizational  boundary  (e.g.,  financial  management, 
acquisition  management,  or  overseas-presence  foreign  affairs  activities). 


An  architecture  describes  the  enterprise’s  operations  in  both  (1)  logical 
terms,  such  as  interrelated  functions,  information  needs  and  flows,  work 
locations,  and  system  applications,  and  (2)  technical  terms,  such  as 
hardware,  software,  data,  communications,  security  attributes,  and 
performance  standards.  It  provides  these  perspectives  for  both  the 
enterprise’s  current,  or  “as  is,”  environment  and  its  target,  or  “to  be,” 
environment;  it  also  provides  an  IT  capital  investment  road  map  for 
moving  between  the  two  environments. 


The  development,  implementation,  and  maintenance  of  enterprise 
architectures  are  recognized  hallmarks  of  successful  public  and  private 
sector  organizations.  Managed  properly,  an  enterprise  architecture  can 
clarify  and  help  optimize  the  interdependencies  and  interrelationships 
among  an  organization’s  business  operations  and  the  underlying  IT 
infrastructure  and  applications  that  support  these  operations.  Our 
experience  with  federal  agencies  has  shown  that  attempting  to  define  and 
build  major  IT  systems  without  first  completing  an  enterprise  architecture 
often  results  in  systems  that  are  duplicative,  not  well  integrated, 
unnecessarily  costly  to  maintain  and  interface,  and  not  effectively 
optimizing  mission  performance. 


Federal  CIO  Council  guidance  defines  a  set  of  recognized  key  practices 
(management  structures  and  processes)  for  developing  and  implementing 
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an  enterprise  architecture  that  are  employed  by  successful  public  and 
private  sector  organizations.  “  Among  other  things,  these  practices  include 
the  following: 

•  Because  the  architecture  is  a  corporate  asset  for  systematically  managing 
institutional  change,  the  head  or  leader  of  the  enterprise  should  support 
and  sponsor  the  architecture,  giving  it  a  clear  mandate  in  the  form  of  an 
enterprise  policy  statement.  Such  support  is  crucial  to  gaining  the 
commitment  of  all  organizational  components  of  the  enterprise,  all  of 
whom  should  participate  in  developing  and  implementing  the  enterprise 
architecture. 

•  The  enterprise  architecture  effort  should  be  directed  and  overseen  by  an 
executive  body,  empowered  by  the  head(s)  of  the  organization(s),  with 
members  who  represent  all  stakeholder  organizations  and  have  the 
authority  to  commit  resources  and  to  make  and  enforce  decisions  for  their 
respective  organizations. 

•  The  enterprise  architecture  effort  should  be  led  by  a  Chief  Architect  who 
reports  to  the  enterprise  CIO,  and  it  should  be  managed  as  a  formal 
program.  A  formal  program  entails  creating  a  program  office;  committing 
core  staff;  implementing  a  program  management  plan  that  details  work 
breakdown  structure  and  schedules;  allocating  resources  and  tools; 
performing  basic  program  management  functions  (e.g.,  risk  management, 
change  control,  quality  assurance,  configuration  management);  and 
tracking  and  reporting  progress  against  measurable  goals. 

•  The  enterprise  architecture  should  conform  to  a  specified  framework. 

Although  State  modernization  officials  have  acknowledged  the  importance 
of  having  an  overseas-presence  enterprise  architecture  and  using  this 
architecture  to  guide  and  constrain  investment  in  a  common  overseas 
knowledge  management  system,  to  date  substantive  progress  in  this  area 
has  not  occurred.  Currently,  State  and  its  agency  partners  have  yet  to 
either  develop  the  architecture  or  establish  effective  management 
structures  and  processes  for  developing,  maintaining,  and  implementing 
one,  such  as  those  specified  in  federal  CIO  Council  guidance.  More 
specifically,  although  State  established  an  Enterprise  Architecture  Study 
Group  in  December  2000,  the  group  met  only  twice  and  progressed  no 
further  than  drafting  a  charter  and  setting  a  goal  of  completing  the 
architecture  by  November  2001. 


^^Chief  Information  Officer  Council,  A  Practical  Guide  to  Federal  Enterprise  Architecture, 
Version  1.0  (February  2001). 
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On  June  8,  2001,  State  designated  a  Chief  Architect  to  lead  this  effort.  In 
July  2001,  State  reconvened  the  group,  renaming  it  the  Foreign  Affairs 
Systems  Integration  Architectural  Working  Group,  and  set  February  15, 
2002,  as  the  new  goal  for  completing  the  architecture.  Since  reconvening, 
the  working  group  has  held  three  meetings  to  begin  planning  for 
development  of  the  overseas-presence  enterprise  architecture  and 
developed  a  draft  charter.  The  working  group  has  also  begun  surveying  its 
agency  partners  to  obtain  general  information  on  architectural 
requirements  of  overseas  locations.  State’s  ongoing  prototyping  and 
planned  piloting  efforts,  which  are  intended  to  provide  information  to 
better  understand  business  and  technology  requirements  and  alternate 
approaches,  should  facilitate  development  of  the  architecture. 

It  is  essential  that  State  and  its  partner  agencies  effectively  develop, 
maintain,  and  implement  an  enterprise  architecture  for  its  overseas 
modernization  program.  The  federal  CIO  Coimcil  architecture 
management  guide  provides  a  road  map  for  doing  so.  Our  experience  with 
system  modernization  programs  in  other  federal  agencies  has  shown  that 
attempting  to  define  and  build  major  systems  without  first  completing  an 
enterprise  systems  architecture  often  results  in  systems  that  are 
duplicative,  not  well  integrated,  imnecessarily  costly  to  maintain  and 
interface,  and  not  effectively  optimizing  mission  performance.^" 


Disciplined  Acquisition 
Management  Processes 
Can  Further  Reduce 
Program  Risks 


The  Chnger-Cohen  Act  requires  agency  CIOs  to  establish  effective  IT 
management  processes,  such  as  processes  for  acquiring  software.^^  The 
SEI,  recognized  for  its  expertise  in  software  process  controls,  has 
developed  a  model  approach  for  software  acquisition.  This  model  outlines 
the  key  acquisition  process  management  controls  that,  if  implemented 
effectively,  can  greatly  increase  the  chances  of  acquiring  software- 
intensive  systems  that  provide  promised  capabilities  on  time  and  within 
budget.  The  key  processes  are  as  follows: 


^"See,  for  example,  Air  Traffic  Control:  Complete  and  Enforced  Architecture  Needed  for 
FAA  Systems  Modernization  (GAO/AIMD-97-30,  February  3,  1997)  and  Customs  Service 
Modernization:  Architecture  Must  Be  Complete  and  Enforced  to  Effectively  Build  and 
Maintain  Systems  (GAO/AIMD-98-70,  May  5,  1998). 

^'Clinger-Cohen  Act  of  1996,  Public  Law  104-106. 


Page  21 


GAO-02-41  State-Led  Modernization  Initiative 


•  Software  Acquisition  Planning:  Ensures  that  reasonable  planning  for  the 
software  acquisition  is  conducted  and  that  all  aspects  of  the  total  software 
acquisition  effort  are  included  in  these  plans. 

•  Solicitation:  Ensures  that  a  request  for  proposal  that  delineates  a  project’s 
software-related  requirements  is  prepared,  and,  consistent  with  relevant 
solicitation  laws  and  regulations,  that  a  contractor  that  can  most  cost- 
effectively  satisfy  these  requirements  is  selected. 

•  Requirements  Development  and  Management:  Establishes  and  maintains 
a  common  and  unambiguous  definition  of  software  requirements  among 
the  acquisition  team,  the  system  users,  and  the  software  development 
contractor. 

•  Project  Office  Management:  Provides  for  management  of  the  activities 
within  the  project  office  and  supporting  contractors  to  ensure  a  timely, 
efficient,  and  effective  software  acquisition. 

•  Contract  Tracking  and  Oversight:  Ensures  that  the  software  development 
contractor  performs  according  to  the  terms  of  the  contract;  needed 
contract  changes  are  identified,  negotiated,  and  incorporated  into  the 
contract;  and  contractor  performance  issues  are  identified  early,  when 
they  are  easier  to  address. 

•  Evaluation:  Determines  whether  the  acquired  software  products  and 
services  satisfy  contract  requirements  prior  to  acceptance. 

•  Transition  and  Support:  Provides  for  the  effective  and  efficient  “hand-off’ 
of  the  acquired  software  products  to  the  support  organization  responsible 
for  software  maintenance. 

•  Acquisition  Risk  Management:  Identifies  risks  as  early  as  possible  and 
adjusts  the  acquisition  to  mitigate  those  risks. 

Within  these  key  processes,  SEl  identifies  key  practices  that  are  needed  to 
effectively  execute  each  key  process.  Among  others,  these  key  practices 
include:  (1)  designating  responsibility  for  activities  and  assigning  adequate 
resources;  (2)  having  a  written  policy;  (3)  developing,  documenting,  and 
adhering  to  a  plan;  (4)  performing  management  review  activities;  and  (5) 
measuring  the  status  of  key  activities,  and  using  these  measurements  to 
make  decisions. 

According  to  State  program  officials,  the  modernization  program  will  be 
managed  in  accordance  with  State’s  methodology.  Managing  State  Projects 
(MSP).  MSP  specifies  major  project  management  activities  to  be 
performed,  the  products  to  be  generated,  and  the  control  gates  (i.e., 
decision  points)  to  be  used  to  ensure  that  projects  are  ready  for  the  next 
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phase  in  their  life  cycle.  We  analyzed  State’s  MSP  and  found  that  it  is 
consistent  with  best  practice  models,  such  as  SEFs  acquisition  model.^^ 

For  its  near-term  prototyping  and  pilot-testing  activities,  which  do  not 
demand  the  level  of  management  process  control  rigor  and  discipline 
associated  with  acquiring  a  system  solution  intended  for  operational  use. 
State  is  employing  adequate  acquisition  management  process  controls.  For 
example.  State  has  defined  a  work  breakdown  structure  and  schedule  to 
guide  near-term  activities.  Further,  State  CIO  officials  told  us  that 
management  plans  relating  to  key  process  areas  have  been  drafted, 
although  they  did  not  provide  us  with  copies  of  the  draft  plans,  as  we 
requested.  According  to  State,  these  plans  will  be  available  when  finalized 
and  approved.  These  officials  also  told  us  that  program  management  is 
using  two  automated  tools  to  track  the  status  of  program  activities.  We 
verified  that  State  has  been  using  one  of  these  tools  to  track  program- 
related  tasks.^^ 

According  to  State,  it  has  not  yet  established  acquisition  management 
controls  for  the  future  operational  version  of  the  overseas-presence 
knowledge  management  system  because  it  does  not  have  congressional 
authorization  to  plan  beyond  the  pilot  project  for  this  initiative.  Beyond 
intentions  to  employ  those  controls  already  in  place  for  key  processes  for 
the  prototype  and  pilot  efforts  (i.e.,  designating  a  responsible  official  and 
having  a  written  policy).  State  has  important  practices  that  will  need  to  be 
established  before  it  is  ready  to  acquire  the  operational  system.  Table  3 
summarizes  the  key  process  areas  for  managing  acquisition  of  the 
operational  version  of  the  system  that  already  exist  and  those  that  will 
need  to  be  established. 


^^Based  on  a  comparison  of  the  MSB’s  major  process  areas  with  SEFs  key  processes. 
^^We  did  not  pursue  verification  of  State’s  use  of  the  other  tool. 
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Table  3:  Summary  of  Needed  Process  Controls  for  Managing  the  Acquisition  of  the  Operational  Version  of  the  System 


Perform 

Measure  and 

Process  area 

Designate 

responsible 

official 

Assign 

adequate 

resources 

Have  a 
written 
policy” 

Document  a 
plan  outlining 
activities 

management 
review  of 
activities 

report  on 
status  of 
activities 

Software  acquisition 
planning 

V 

0 

V 

0 

0 

0 

Soiicitation 

V 

0 

V 

0 

0 

0 

Requirements 
deveiopment  and 
management 

V 

0 

V 

0 

0 

0 

Project  office 
management 

V 

0 

V 

0 

0 

0 

Contract  tracking  and 
oversight 

V 

0 

V 

0 

0 

0 

Evaluation 

V 

0 

V 

0 

0 

0 

Transition  and  support 

V 

0 

V 

0 

0 

0 

Acquisition  risk 
management 

V 

0 

V 

0 

0 

0 

V — Satisfied 

0 — Not  satisfied 


"Based  on  State’s  plans  to  adhere  to  its  Foreign  Affairs  Manual,  which  requires  implementation  of  its 
MSP  methodology. 

Source:  State. 


It  is  imperative  that  State  ensures  that  it  fully  implements  key  acquisition 
management  process  controls,  as  defined  in  its  MSP  and  the  SEI  model,  on 
its  overseas  knowledge-management  modernization  program  before 
acquiring  the  operational  system.  Our  reviews  of  other  agency 
modernization  efforts  have  shown  that  failure  to  implement  rigorous  and 
disciplined  acquisition  processes  on  a  given  program  or  project  can  lead  to 
systems  that  do  not  perform  as  intended,  cost  more  than  expected,  and  are 
not  delivered  on  time.^^ 


^‘'See,  for  example,  Customs  Service  Modernization:  Serious  Management  and  Technical 
Weaknesses  Must  Be  Corrected  CGAO/AIMD-99-41,  February  26,  1999)  and  Land 
Management  Systems:  BLM’s  Actions  to  Improve  Information  Technology  Management 
(GAO-01-282,  February  27,  2001). 


Page  24 


GAO-02-41  State-Led  Modernization  Initiative 


Conclusions 


The  scope  of  the  near-term  activities  for  State’s  overseas-presence 
knowledge  management  modernization  program,  as  described  by  State 
CIO  officials  and  described  in  draft  plans,  is  appropriately  limited  to 
system  prototyping  and  associated  pilot  testing  of  a  version  of  the 
operational  system.  In  light  of  this  scope,  as  well  as  State’s  leadership  role 
in  managing  and  funding  these  near-term  efforts  and  its  efforts  to  date  to 
bring  its  agency  partners  together,  there  is  currently  an  adequate  level  of 
(1)  explicit  definition  and  understanding  surrounding  the  overseas- 
presence  agencies’  respective  responsibilities  and  accountabilities,  (2) 
agency  involvement  in  the  program,  and  (3)  State’s  management  control 
discipline. 

However,  before  State  can  effectively  acquire  and  deploy  the  operational 
knowledge  management  system,  it  needs  to  more  explicitly  define 
stakeholder  agencies’  program  management  and  funding  roles,  as  well  as 
establish  more  rigorous  and  disciplined  management  controls. 
Accordingly,  it  is  critical  for  State  to  also  move  aggressively  in  the  near 
term  to  (1)  clearly  define,  through  written  agreements,  the  respective 
management  and  funding  responsibilities  and  accountabilities  of  the 
program’s  agency  partners  and  (2)  institute  key  modernization 
management  controls,  including  incremental  investment  management 
practices,  enterprise  architecture  governance,  and  software  acquisition 
management  rigor  and  discipline.  While  State  has  verbally  committed  to 
addressing  these  areas,  much  remains  to  be  done  before  State  and  its 
agency  partners  will  be  ready  to  proceed  beyond  prototyping  and  pilot 
testing  and  begin  acquiring  the  system  solution  intended  for  operational 
deployment  and  use.  Attempting  to  acquire  the  operational  system 
solution  without  these  controls  risks  not  delivering  needed  system 
capabilities  on  time  and  within  budget. 


Recommendations  for 
Executive  Action 


To  provide  for  an  explicit  definition  and  understanding  of  the  respective 
roles  of  all  overseas-presence  modernization  agency  stakeholders,  we 
recommend  that  the  Secretary  of  State  develop  and  submit  a  proposal  for 
0MB  and/or  congressional  action,  whichever  the  Secretary  deems 
appropriate,  within  60  days  of  the  date  of  this  report  that  clearly  defines 
and  assigns  responsibilities  and  accountabilities  for  each  overseas- 
presence  agency  regarding  the  knowledge  management  system 
modernization  program.  We  further  recommend  that  the  Secretary  direct 
the  Under  Secretary  for  Management  to  ensure  that  this  proposal  (1)  sets 
expectations  for  each  overseas-presence  agency,  including  the  State 
Department,  relative  to  modernization  management  and  funding  and  (2)  is 
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coordinated  with  and  agreed  to  by  the  Under  Secretary’s  counterparts  in 
each  of  the  partner  agencies. 

To  ensure  that  key  system  modernization  management  controls  are 
implemented  before  the  operational  system  is  acquired,  we  recommend 
that  the  Under  Secretary  of  State  for  Management  direct  State’s  CIO,  as 
chair  of  the  Interagency  Technology  Subcommittee,  to  ensure  that  the 
subcommittee  allows  only  architecturally  compliant  knowledge 
management  system  investments  to  be  approved  and  funded,  unless  the 
subcommittee  issues  a  written  waiver  in  response  to  a  written 
justification.  Furthermore,  we  recommend  that  the  Under  Secretary  direct 
the  CIO  to  ensure  that  architecturally  compliant  investments  in  the  system 
be  made  incrementally.  To  this  end,  the  CIO  should  ensure  that  processes 
and  structures  are  established  for  (1)  acquiring  the  system  in  a  series  of 
smaller  system  increments;  (2)  individually  justifying  investment  in  each 
increment  on  the  basis  of  costs,  benefits,  and  risks;  and  (3)  monitoring 
actual  benefits  achieved  and  costs  incurred  on  ongoing  increments  and 
modifying  subsequent  increments  to  reflect  lessons  learned. 

To  further  ensure  that  key  management  controls  are  implemented  on  the 
overseas-presence  knowledge  management  system  modernization,  we 
recommend  that  the  Secretary  of  State  designate  as  a  modernization 
program  priority  the  development,  implementation,  and  maintenance  of  an 
overseas  foreign-affairs  presence  enterprise  architecture.  To  this  end,  we 
recommend  that  the  Secretary  direct  the  Under  Secretary  for  Management 
to  establish  an  enterprise  architecture  steering  committee,  chaired  by  the 
Under  Secretary  and  composed  of  the  Under  Secretary’s  counterparts 
from  the  other  overseas-presence  agencies.  We  also  recommend  that  this 
steering  committee  be  assigned  responsibility  and  accountability  for 
ensuring  that  a  complete  and  approved  enterprise  architecture  is 
developed  and  available  to  guide  and  constrain  overseas-presence  system 
modernization  efforts.  To  assist  the  steering  committee,  we  recommend 
that  the  Under  Secretary  direct  the  State  CIO  to  establish  an  enterprise 
architecture  project  office,  headed  by  the  recently  named  Chief  Architect, 
to  develop  and  maintain  the  enterprise  architecture  and  submit 
appropriate  versions  of  this  architecture  to  the  steering  committee  for 
approval.  We  further  recommend  that  the  State  CIO,  at  a  minimum,  ensure 
that  the  project  office  (1)  include  staff  from  each  of  the  overseas-presence 
agencies;  (2)  integrate  the  architecture  with  these  agencies’  respective 
operational  and  systems  environments;  and  (3)  adhere,  as  appropriate, 
with  the  federal  CIO  Council’s  published  guidance  on  managing  enterprise 
architectures. 
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To  further  address  State’s  need  to  implement  modernization  management 
controls,  we  recommend  that  the  Under  Secretary  for  Management  direct 
the  State  CIO  to  ensure  that  the  overseas-presence  system  modernization 
office,  consistent  with  State’s  project  management  requirements  and 
recognized  models  for  effective  software  acquisition,  establishes  and 
implements  rigorous  and  disciplined  acquisition  processes,  including 
processes  for  acquisition  planning,  contractor  solicitation,  requirements 
development  and  management,  contractor  tracking  and  oversight,  testing 
and  evaluation,  transition  to  support  the  acquired  system,  and  risk 
management. 

Until  these  recommended  modernization  management  controls  are  in 
place,  we  also  recommend  that  the  Secretary  limit  further  investment  in 
the  overseas-presence  knowledge  management  modernization  program  to 
(1)  conducting  ongoing  prototyping  and  limited  pilot  testing  and  (2) 
establishing  needed  modernization  management  controls. 


Agency  Comments 
and  Our  Evaluation 


In  written  comments  on  a  draft  of  this  report  (reprinted  in  appendix  11), 
State’s  Acting  Chief  Financial  Officer  agreed  with  our  findings  and 
conclusions  that  its  management  controls  over  near-term  prototype  and 
pilot-testing  activities  are  adequate,  but  disagreed  with  our  finding  and 
conclusion  that  State’s  existing  controls  are  not  adequate  for  effectively 
managing  the  more  demanding  next  phase  of  the  program — acquisition 
and  deployment  of  the  operational  system.  Furthermore,  State  disagreed 
with  our  recommendations  for  instituting  stronger  management  controls 
before  proceeding  to  this  next  phase.  According  to  State,  our 
recommendations  should  be  deleted  from  the  report  because  (1)  we 
reported  that  management  controls  over  near-term  activities  are  adequate 
and  thus  corrective  actions  are  not  needed;  (2)  except  for  clearly  defining 
and  agreeing  to  all  agencies’  roles  and  responsibilities.  State’s  existing 
management  controls  are  adequate  for  acquiring  and  deploying  the 
operational  system;  and  (3)  the  lead  agency  for  the  next  phase  of  the 
program  is  uncertain  and  thus  the  recommendations  are  premature.  At  the 
same  time.  State  said  that  most  of  the  recommendations  might  more 
reasonably  be  submitted  as  suggestions  for  consideration  to  the  lead 
agency  for  the  next  phase  of  the  program. 

We  do  not  agree  with  State’s  comment  that  its  existing  management 
controls  are  adequate  for  the  next  phase  of  the  program.  As  stated  in  our 
report,  the  nature  and  purpose  of  system  prototyping  and  pilot  testing  are 
fundamentally  different;  thus,  the  associated  risks  are  less  significant  than 
those  related  to  acquiring  a  system  that  is  to  be  operationally  deployed.  As 
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a  result,  it  is  widely  recognized  that  the  level  of  management  control  to 
effectively  put  in  place  an  operational  system,  particularly  a  system  like 
the  overseas  knowledge-management  system  that  involves  multiple 
agencies,  is  greater  than  is  needed  to  prototype  or  pilot  test  the  system. 
Thus,  for  example,  while  State’s  lack  of  an  enterprise  architecture  to  guide 
and  constrain  near-term  prototype  and  pilot-testing  activities  does  not 
introduce  significant  program  risk,  it  would  introduce  considerable  risk 
for  State  to  begin  the  next  program  phase  without  this  architecture. 
Similarly,  while  State’s  approach  to  managing  its  investment  in  near-term 
activities  is  adequate,  effectively  acquiring  and  deploying  the  operational 
system  will  require  rigorous  management  processes  that,  for  example, 
provide  for  incremental  economic  justification  and  measurement  of 
system  retum-on-investment,  and  provide  for  fully  implementing 
disciplined  acquisition  management  practices  in  such  areas  as  managing 
system  requirements. 

We  also  do  not  agree  with  State’s  rationale  for  deleting  the  report’s 
recommendations  because  these  recommendations  (1)  are  not  intended  to 
correct  problems  with  management  of  the  prototype  and  pilot  activities,  as 
State  asserts,  but  rather  to  proactively  provide  a  recommended  road  map 
for  how  to  successfully  manage  the  next  phase  of  the  program;  (2)  are 
consistent  with  our  conclusions  that  existing  management  controls  are  not 
adequate  for  the  next  phase  of  the  program;  and  (3)  recognize  that  prudent 
planning  requires  that  agency  roles  and  responsibilities  for  the  next  phase 
of  the  program,  including  designation  of  the  lead  agency,  be  clearly 
defined  and  agreed  to  during  this  phase  to  better  ensure  program 
continuity  and  to  avoid  unnecessary  delays.  Accordingly,  the 
recommendations  are  both  appropriate  and  warranted. 

State’s  written  comments,  along  with  our  responses,  are  reproduced  in 
appendix  11. 


We  are  sending  copies  of  this  report  to  the  Chairman  and  Ranking 
Minority  Members  of  the  Senate  Committee  on  Governmental  Affairs; 
House  Committee  on  Government  Reform;  Senate  Committee  on  Foreign 
Relations;  House  Committee  on  International  Relations;  the  Subcommittee 
on  Commerce,  Justice,  State,  and  the  Judiciary,  Senate  Committee  on 
Appropriations;  and  the  Subcommittee  on  Commerce,  Justice,  State,  the 
Judiciary  and  Related  Agencies,  House  Committee  on  Appropriations.  We 
are  also  sending  copies  to  the  Director,  Office  of  Management  and  Budget; 
the  Secretary  of  State;  the  Under  Secretary  for  Management;  and  the  Chief 
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Information  Officer,  the  Department  of  State.  Copies  will  be  made 
available  to  others  upon  request. 


If  you  have  any  questions  on  matters  discussed  in  this  report,  please  call 
me  at  (202)  512-3439  or  Cynthia  Jackson,  Assistant  Director,  at  (202)  612- 
5086.  We  can  also  be  reached  by  e-mail  at  hiter@gao.gov  and 
iacksonc@gao.gov,  respectively.  Key  contributors  to  this  assignment  are 
listed  in  appendix  III. 

Sincerely  yours. 


Randolph  C.  Hite 

Director,  Information  Technology  Systems  Issues 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


Our  objectives  were  to  determine  (1)  what  State’s  plans  are  for  a 
multiagency,  common  platform,  knowledge  management  system,  and  (2) 
whether  State  has  effective  controls  in  place  for  acquiring  this  system,  that 
is,  defined  roles  for  State  and  its  agency  partners  and  effective  controls 
over  its  investment,  enterprise  architecture,  and  system  acquisition 
management  processes. 

To  determine  State’s  plans  for  the  system,  we  reviewed  State  budget 
submissions,  relevant  congressional  direction  (hearing  testimony, 
transcript,  and  related  legislation),  the  results  of  our  prior  review, the 
Overseas  Presence  Advisory  Panel  report  and  recommendations,^®  and 
available  State  modernization  program  management  plans  and  documents. 
Additionally,  we  interviewed  State  modernization  program  officials, 
including  the  Deputy  Chief  Information  Officer  (C10)/Chief  Knowledge 
Officer,  Deputy  Chief  Knowledge  Officer,  and  the  Director  for  Standards, 
Policy,  Planning,  and  Architecture,  to  discuss  State’s  near-  and  long-term 
plans,  as  well  as  representatives  of  the  Overseas  Presence  Committee  and 
the  Interagency  Technology  Subcommittee  to  discuss  the  status  of  these 
plans  and  activities,  and  inter-  and  intra-agency  coordination  concerns.  To 
confirm  the  extent  to  which  State  had  engaged  its  agency  partners,  we 
surveyed  State’s  eight  agency  partners  to  determine  whether  they  (1)  were 
satisfied  with  State’s  efforts  to  engage  them  in  the  program,  (2)  had 
reviewed  and  approved  the  program  plans  and  related  documents,  and  (3) 
believed  that  State  was  adequately  addressing  their  requirements  and/or 
information  needs.  In  surveying  State’s  agency  partners,  we  contacted  and 
received  responses  from  those  individuals  designated  by  their  respective 
agencies  as  the  authorized  representatives  for  this  initiative. 

To  determine  whether  State  has  effective  controls  in  place  for  acquiring 
the  system,  we  did  the  following. 


U.S.  Department  of  State:  The  Budget  in  Brief-Fiscal  Year  2002',  U.S.  Department  of 
State:  Congressional  Presentation  Document — Fiscal  Year  2001  Budget  Volume  I; 

Written  Testimony  of  Fernando  Burbano,  Department  of  State  Chief  Information  Officer, 
presented  to  The  House  Committee  on  International  Relations  (June  22,  2000);  Oversight  of 
the  State  Department:  Technology  Modernization  and  Computer  Security,  Hearing  before 
the  Committee  on  International  Relations,  House  of  Representatives,  Serial  No.  106-171, 
June  22,  2000;  Office  of  Management  and  Budget  (0MB)  Circular  A-130,  Management  of 
Federal  Information  Resources  (November  30,  2000);  and  Foreign  Affairs:  Effort  to 
Upgrade  Information  Technology  Overseas  Faces  Formidable  Challenges 
(GAO/T-AIMD/NSIAD-00-214,  June  22,  2000). 

^^America’s  Overseas  Presence  in  the  21st  Century:  The  Report  of  the  Overseas  Presence 
Advisory  Panel,  U.S.  Department  of  State  (November  1999). 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


•  To  determine  whether  State  and  its  agency  partners’  roles,  responsibilities, 
and  accountabilities  had  been  formally  defined  for  each  phase  of  the 
program  (e.g.,  prototype,  pilot),  we  reviewed  documentation  provided  by 
State  (e.g.,  briefing  slides,  meeting  minutes,  working  group  charters), 
interviewed  State  officials,  and  surveyed  State’s  eight  agency  partners,  as 
described  above. 

•  To  determine  State  and  its  agency  partners’  plans  for  managing  investment 
in  the  system,  we  reviewed  relevant  legislative  requirements  and 
associated  0MB  guidance,^^  as  well  as  applicable  State  policy  and 
guidance,  on  incremental  investment  management. We  then  obtained  and 
analyzed  documentation  outlining  State’s  investment  review  process  for 
selecting,  controlling,  and  evaluating  projects  since  plans  are  currently  for 
State  to  fully  fund  the  project  (prototype  and  pilot).  Because 
documentation  on  State’s  investment  management  activities  to  date  and 
future  plans  for  this  system  were  not  available,  we  interviewed  officials 
from  the  modernization  program  office  [Foreign  Affairs  Systems 
Integration  (FASl)  Office]  and  Deputy  CIO  for  Architecture,  Planning,  and 
Regulations  Office  to  determine  if  State  and  its  partners  planned  to  use  an 
incremental  investment  management  strategy.  We  then  compared 
available  information  on  State’s  investment  management  plans  against 
relevant  federal  requirements  and  guidance  to  identify  if  any  variances 
existed. 

•  To  determine  whether  an  enterprise  architecture  exists  for  the  system  to 
guide  and  constrain  investment,  we  reviewed  relevant  legislative 
requirements  and  associated  0MB  guidance,^'’  as  well  as  federal  CIO 
Council  published  guidance^”  and  applicable  State  policy  and  guidance,^^ 
on  developing,  maintaining,  and  implementing  an  enterprise  architecture. 


^'^Clinger-Cohen  Act  of  1996,  Public  Law  104-106;  0MB  Circular  A-130  (November  30,  2000); 
Capital  Programming  Guide,  Version  1.0,  0MB  Circular  A-11,  part  3  (July  1997),  pp.  545- 
572;  and  Capital  Programming  Guide,  Version  1.0,  OMB  Circular  A-11,  part  3, 
Supplement,  Planning,  Budgeting,  and  Acquisition  of  Capital  Assets  (July  2000),  pp.  35- 
37. 

^U.S.  Department  of  State  Foreign  Affairs  Manual,  Section  5  FAM  626.2:  Incremental 
(February  4,  2000,  update). 

^^Clinger-Cohen  Act  of  1996,  Public  Law  104-106  and  OMB  Circular  A-130  (November  30, 
2000). 

^'’Chief  Information  Officer  Council,  A  Practical  Guide  to  Federal  Enterprise  Architecture, 
Version  1.0  (February  2001). 

^'^U.S.  Department  of  State  Foreign  Affairs  Manual,  Section  5  FAM  115:  IT  Management 
Oversight  Objectives  (January  23,  2001,  update). 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


We  then  reviewed  available  documentation,  including  plans  and 
management  structures,  a  charter,  process  controls,  and  architectural 
artifacts,  as  well  as  interviewing  FASl  Program  Office  officials,  including 
the  Director  for  Standards,  Policy,  Planning,  and  Architecture,  to 
determine  the  state  of  any  enterprise  architecture  development  and 
implementation  efforts.  We  then  compared  the  available  information  to 
relevant  federal  requirements  and  guidance  to  identify  if  any  variances 
existed. 

•  To  determine  whether  State  and  its  agency  partners  have  the  needed 
capabilities  to  effectively  acquire  the  system,  we  obtained  and  analyzed 
documentation  on  agencywide  policies  and  procedures  governing  system 
acquisition  efforts,  which  State  refers  to  as  its  Managing  State  Projects 
methodology.^^  We  then  reviewed  available  best  practice  models  on  system 
and  software  acquisition  management  (e.g.,  the  Software  Engineering 
Institute  (SEl)  modeP^)  and  compared  the  primary  components  of  the 
Managing  State  Projects  methodology  against  the  primary  components  of 
SETs  model  to  determine  if  any  variances  existed.  SEFs  primary 
components  include  software  acquisition  planning,  solicitation, 
requirements  development  and  management,  project  office  management, 
contractor  tracking  and  oversight,  evaluation,  transition  and  support,  and 
risk  management.  Because  we  found  that  the  primary  components  of 
State’s  methodology  were  consistent  with  best  practices,  we  obtained  and 
reviewed  available  program  management  and  system  documentation  and 
interviewed  FASl  Program  Office  officials  to  determine  the  extent  to 
which  the  methodology  was  being  implemented  on  the  program. 

We  performed  our  work  at  State  Department  headquarters  in  Washington, 
D.C.,  from  October  2000  through  November  2001  in  accordance  with 
generally  accepted  government  auditing  standards. 


^^Managing  State  Projects,  U.S.  Department  of  State  (January  19,  1999). 

^^Software  Acquisition  Capability  Maturity  Model™  (SA-CMM®).  Capability  Maturity 
Model™  is  a  service  mark  of  Carnegie  Mellon  University,  and  CMM®  is  registered  in  the 
U.S.  Patent  and  Trademark  Office. 
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Appendix  II:  Comments  From  the 
Department  of  State 


Note:  GAO  comments 
supplementing  those  in 
the  report  text  appear  at 
the  end  of  this  appendix. 


See  comment  1 . 


United  States  Department  of  State 

Chief  Financial  Officer 
Washington,  D.C.  20520-7427 

OCT  i  7  200! 


Dear  Ms .  Westin : 


We  appreciate  the  opportunity  to  review  your  draft  report, 
"INFORMATION  TECHNOLOGY:  State  Department  Led  Overseas 
Modernization  Program  Faces  Management  Challenges,"  GAO-02-41, 
GAO  Job  Code  310204. 

The  enclosed  Department  of  State  comments  addressing  the 
GAO  report's  three  conclusions,  three  assertions,  and  twelve 
recommendations  are  provided  for  incorporation  with  this  letter 
as  an  appendix  to  the  final  report.  The  report  recommendations 
are  listed  in  their  entirety  in  an  appendix  to  State's  comments. 

In  summary,  the  Department  of  State  agrees  with  GAO' s 
conclusions.  The  comments  address  State's  concerns  with  GAO' s 
assertions  and  requests  elimination  of  the  recommendations. 

To  discuss  this  response,  please  contact  Roy  Standing, 
Deputy  Chief  Information  Officer  for  Architecture,  Planning,  and 
Regulations,  Bureau  of  Information  Resource  Management,  on  (202) 
647-3910. 

Sincerely, 


Enclosure : 


As  stated. 


J.  Eisenhart 
Acting 


GAO/IT  -  Mr.  Willemssen 
State/OIG  -  Mr.  Atkins 
State/IRM  -  Mr.  Standing 


Ms.  Susan  S.  Westin, 

Managing  Director, 

International  Affairs  and  Trade, 

U.S.  General  Accounting  Office. 
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See  comment  2. 


See  comment  3. 


INFORMATION  TECHNOLOGY:  State  Department  Led  Overseas 
Modernization  Program  Faces  Management  Challenges 
(GAO-02-41 ,  GAO  Code  31 0204) 


Department  of  State  Comments  on  GAO  Draft  Report 


State’s  herein  provides  comments  on  GAO's  draft  report.  GAO  provided  three 
conciusions,  three  assertions  and  twelve  recommendations.  State's  response  is 
organized  as  foliows: 

Document:  State  Response  to  GAO  Draft  Report 

Appendix  A  GAO  Recommendations 

Executive  Summary 

The  Department  of  State  appreciates  this  opportunity  to  respond  to  the  GAO 
draft  report.  We  have  reviewed  the  draft  report  and  agree  with  the  overall 
findings  of  our  efforts  on  the  prototype  and  pilot  for  an  interagency  foreign  affairs 
knowledge  management  system.  The  purpose  of  the  system  is  to  provide  an 
environment  for  foreign  affairs  agencies  with  overseas  presence  to  more 
effectively  communicate  on  an  unclassified  network.  State  has  and  continues  to 
review  alternative  approaches. 

GAO  reviewed  the  near-term  State  efforts  to  develop  a  prototype  and  pilot  and 
found  that  State  has  adequately  managed  and  funded  near-term  efforts.  GAO’s 
statement  of  adequacy  applies  to  the  foreign  affairs  agencies’  roles, 
responsibilities  and  accountabilities;  State’s  involvement  in  the  system;  and 
State’s  management  controls. 

Therefore,  in  light  of  your  findings,  State  was  alarmed  to  see  12  corrective  action 
recommendations  at  the  end  of  the  report.  State  was  unable  to  discover  a 
relationship  between  the  substance  of  the  report,  its  conclusions  and  the 
recommendations  for  corrective  action.  State  is  recommending  the  deletion  of 
these  recommendations  from  the  final  report. 


1 
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Appendix  II:  Comments  From  the  Department 
of  State 


INFORMATION  TECHNOLOGY:  State  Department  Led  Overseas 
Modernization  Program  Faces  Management  Challenges 
(GAO-02-41,  GAO  Code  310204) 


Section  I;  GAO  Conclusions 


A.  The  current  level  of  explicit  definition  and  understanding  surrounding  the 
overseas  presence  agencies’  respective  responsibilities  and  accountabilities 
is  adequate  for  this  near-term  effort. 

State  agrees  with  this  conclusion  and  has  no  comments. 


B,  The  current  level  of  agency  involvement  in  the  program  is  adequate  for  this 
near-term  effort. 

State  agrees  with  this  conclusion  and  has  no  comments. 


C.  The  current  level  of  State's  management  control  discipline  is  adequate  for  this 
near-term  effort. 

State  agrees  with  this  conclusion  and  has  no  comments. 
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of  State 


See  comment  4. 


See  comment  5. 


See  comment  6. 


See  comment  7. 
See  comment  8. 


INFORMATION  TECHNOLOGY:  State  Department  Led  Overseas 
Modernization  Program  Faces  Management  Challenges 
(GAO-02-41,  GAO  Code  310204) 


Section  II:  GAO  Assertions  and  Recommendations 


A.  GAO  asserts  that  State  must  employ  more  rigorous  management  controls  for 
the  acquisition  and  deployment  efforts  beyond  the  prototype  and  pilot  testing. 

State  agrees  that  rigorous  management  controis  are  necessary;  however,  State 
disagrees  that  existing  controls  are  not  adequate  for  use  beyond  the  prototype 
and  pilot  testing.  This  project  has  used  State's  existing  management  controls  for 
all  activities  within  the  prototype  and  pilot  project.  State  has  rigorous 
management  controls  in  place  and  operating  as  intended.  State’s  capital 
planning  and  investment,  enterprise  architecture  and  managing  state  projects 
processes  are  intertwined  and  each  project  manager  must  consider  and 
incorporate  these  processes  during  the  course  of  each  and  every  project.  State 
is  involving  the  procurement  executive  to  provide  earlier  and  more  thorough 
consideration  of  acquisition  issues  for  projects.  If  State  is  the  lead  agency  for 
efforts  beyond  the  prototype  and  pilot,  then  State’s  management  controls  will  be 
available  for  utilization. 

State  has  not  yet  proposed  program  and  management  plans  for  any  efforts 
beyond  the  prototype  and  pilot.  State’s  Chief  Information  Officer  (CIO)  briefed  a 
professional  staff  member  of  the  Chairman  of  the  Subcommittee  on  Commerce, 
Justice,  State,  the  Judiciary  and  Related  Agencies,  Senate  Committee  on 
Appropriations  on  the  prototype  and  pilot  activities  in  March  2001 . 

State’s  CIO  briefed  the  same  staff  member  again  on  October  5,  2001 ,  concerning 
what  State  and  other  foreign  affairs  agency  partners  have  accomplished  and 
what  current  and  planned  activities  will  be  done  to  complete  the  prototype  and 
pilot  activities.  State  plans  to  conclude  the  project  by  preparing  a  prototype  and 
pilot  evaluation  report  and  developing  lessons  learned.  State  management 
processes  require  this  and  the  information  will  be  needed  by  the  lead  agency  for 
worldwide  deployment. 

GAO  recommendations  3  through  10  (see  Appendix  A)  relate  to  this  assertion. 

As  stated,  the  recommendations  mandate  specific  organizational  structure  and 
methodologies.  In  addition,  these  recommendations  require  specific  actions  of 
the  Secretary  of  State,  Under  Secretary  of  State  for  Management,  and  Under 
Secretaries  for  Management  [equivalents]  of  the  partnering  foreign  affairs 
agencies. 

Since  no  deployment  project  proposal  has  been  prepared,  GAO’s  corrective 
action  recommendations  appear  to  be  unsupported.  Recognizing  GAO’s  role  in 


3 


Page  36 


GAO-02-41  State-Led  Modernization  Initiative 


Appendix  II:  Comments  From  the  Department 
of  State 


See  comment  8. 


INFORMATION  TECHNOLOGY:  State  Department  Led  Overseas 
Modernization  Program  Faces  Management  Challenges 
(GAO-02-41,  GAO  Code  310204) 


reviewing  “Best  Practices”  and  lessons  learned  from  other  projects,  they  might 
more  reasonably  be  submitted  as  suggestions  for  consideration  by  the  lead 
agency  for  worldwide  depioyment.  State  recommends  that  GAO 
recommendations  3  through  10  be  deleted  from  the  final  report. 
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B.  GAO  asserts  that  State  and  its  agency  partners  have  not  fully  and  formally 
defined  and  agreed  to  roles,  responsibilities  and  accountabilities  for  program 
funding  and  management  [and  must  do  so]  before  moving  beyond  planned 
near-term  activities. 


State  disagrees.  State  is  aware  that  more  formal  processes  will  be  needed 
before  worldwide  deployment  but  GAO  recognized  State's  near-term  roles, 
responsibilities  and  accountabilities  as  adequate.  State  has  informed  and 
provided  documentation  to  GAO  on  numerous  occasions  that  the  present 
structure  for  dealing  with  other  foreign  affairs  agency  partners  is  working  and 
agency  partners  have  not  informed  State  that  a  more  formal  process  is  needed. 

GAO  proposed  two  recommendations  (numbers  1  and  2,  see  Appendix  A)  that 
deal  with  defining  roles,  responsibilities  and  accountabilities  for  foreign  affairs 
agencies  including  State.  As  stated,  the  recommendations  mandate  specific 
timeframes  and  methodologies.  In  addition,  these  recommendations  require 
specific  actions  of  the  Secretary  of  State,  Under  Secretary  of  State  for 
Management,  Under  Secretaries  for  Management  [equivalents]  of  the  Partnering 
foreign  affairs  agencies,  and  State’s  CIO. 

The  Department  of  State  cannot  find  a  basis  in  the  GAO  Report  to  support  the 
necessity  of  these  corrective  action  recommendations.  While  formalized  roles 
and  responsibilities  will  be  necessary  for  a  deployment  project,  these  should  be 
addressed  as  part  of  a  deployment  project. 

State  recommends  that  assertion  B  be  revised.  The  assertion  should  be  focused 
on  the  need  to  formally  define  roles,  responsibilities  and  accountabilities  at  a 
level  adequate  to  minimize  risks  in  a  worldwide  deployment  project.  The 
reference  to  existing  insufficiency  of  definition  of  roles,  responsibilities  and 
accountabilities  should  be  eliminated. 

State  recommends  that  GAO  recommendations  1  and  2  be  deleted  from  the  final 
report. 
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C.  GAO  asserts  that  State  needs  disciplined  acquisition  management  processes 
for  future  worldwide  deployment  of  an  operational  system. 


State  agrees  that  disciplined  acquisition  management  processes  will  be 
necessary  for  future  worldwide  deployment  of  an  operational  system;  however, 
State's  existing  acquisition  management  processes  are  disciplined  and  rigorous. 
These  controis  are  in  piace  and  operating  as  intended.  Volume  6  of  the  Foreign 
Affairs  Manual  implements  the  legislative  requirement  of  acquisition  management 
and  processes  that  State  personnel  must  follow.  These  requirements  are  in 
addition  to  Volume  5  covering  project  management.  Every  project  manager  must 
and  will  use  these  processes  to  evaluate  each  and  every  project,  including  the 
prototype  and  pilot. 

State  is  continually  refining  and  improving  its  acquisition  management  systems. 

A  detailed  acquisition  review  by  the  procurement  executive  will  be  a  key  aspect 
of  the  functional  reviews  that  take  place  before  funding  decisions  are  made,  to 
ensure  that  the  acquisition  strategy  makes  good  business  sense. 

GAO  made  recommendations  11  and  12  (see  Appendix  A)  relative  to  this 
assertion.  As  stated,  the  recommendations  mandate  the  establishment  of 
rigorous  and  disciplined  acquisition  processes.  Further,  these  recommendations 
require  specific  actions  of  the  Secretary  of  State,  Under  Secretary  of  State  for 
Management,  and  State’s  CIO. 

The  Department  of  State  cannot  find  a  basis  in  the  GAO  Report  to  support  these 
corrective  action  recommendations,  since  the  current  project  is  already  limited  to 
a  prototype  and  pilot  only.  State  has  rigorous  and  disciplined  acquisition 
processes.  Therefore,  recommendation  12  has  no  basis  and  should  be  deleted. 

In  summary,  State  recommends  that  assertion  C  be  revised  to  state  that 
disciplined  acquisition  management  processes  are  a  key  factor  in  minimizing  the 
risks  of  a  future  worldwide  deployment  of  an  operational  system.  State  also 
recommends  that  GAO  recommendations  1 1  and  12  be  deleted  from  the  final 
report. 
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Section  III;  Summary  and  Recommendations 

It  is  the  opinion  of  the  Department  of  State  that  the  assertions  made  by  GAO  in 
the  subject  report  should  be  revised  as  indicated.  Whiie  the  Department  of  State 
recognizes  and  understands  that  foilow-on  efforts  wiil  require  a  comjjrehensive 
review  of  necessary  organizationai  management  and  administrative  processes,  it 
is  premature  for  GAO  to  make  corrective  action  recommendations  in  advance  of 
the  existence  of  program  proposais.  The  Department  of  State  recommends  the 
deietion  of  recommendations  1  through  12. 
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GAO  Recommendations 


GAO 

RECOMMENDATION 

DESCRIPTION 

1 

To  provide  for  an  expiicit  definition  and  understanding 
of  the  respective  roles  of  all  overseas  presence 
modernization  agency  stakeholders,  we  recommend 
that  the  Secretary  of  State  develop  and  submit  a 
proposal  for  0MB  and/or  congressional  action, 
whichever  the  Secretary  deems  appropriate,  within  60 
days  of  this  report  that  clearly  defines  and  assigns 
responsibilities  and  accountabilities  for  each  overseas 
presence  agency  with  respect  to  the  knowledge 
management  system  modernization  program. 

2 

We  further  recommend  that  the  Secretary  direct  the 
Under  Secretary  for  Management  to  ensure  that  this 
proposal  (1)  sets  expectations  for  each  overseas 
presence  agency,  including  the  State  Department, 
relative  to  modernization  management  and  funding 
and  (2)  is  coordinated  with  and  agreed  to  by  the  Under 
Secretary’s  counterpart  in  each  of  the  partner 
agencies. 

3 

To  ensure  that  key  system  modernization 
management  controls  are  implemented  before 
acquiring  the  operational  system,  we  recommend  that 
the  Under  Secretary  of  State  for  Management  direct 
State’s  CIO,  as  chair  of  the  Interagency  Technology 
Subcommittee,  to  ensure  that  the  subcommittee  allows 
only  architecturally  compliant  knowledge  management 
system  investments  to  be  approved  and  funded, 
unless  the  subcommittee  issues  a  written  waiver  in 
response  to  a  written  justification. 

4 

Further,  we  recommend  that  the  Under  Secretary 
direct  the  CIO  to  ensure  that  architecturally  compliant 
investments  in  the  system  be  made  incrementally. 
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GAO 

RECOMMENDATiON 

DESCRiPTiON 

5 

To  this  end,  the  CIO  should  ensure  that  processes  and 
structures  are  established  for  (1)  acquiring  the  system 
in  a  series  of  smaller  system  increments,  (2) 
individually  justifying  investment  in  each  increment  on 
the  basis  of  costs,  benefits,  and  risks,  and  (3) 
monitoring  actual  benefits  achieved  and  costs  incurred 
on  ongoing  increments  and  modifying  subsequent 
increments  to  reflect  lessons  learned. 

6 

To  further  ensure  that  key  management  controls  are 
implemented  on  the  overseas  presence  knowledge 
management  system  modernization,  we  recommend 
that  the  Secretary  of  State  designate  the  development, 
implementation,  and  maintenance  of  an  overseas 
foreign  affairs  presence  enterprise  architecture  as  a 
modernization  program  priority. 

7 

To  this  end,  we  recommend  that  the  Secretary  direct 
the  Under  Secretary  for  Management  to  establish  an 
enterprise  architecture  steering  committee,  chaired  by 
the  Under  Secretary  and  composed  of  the  Under 
Secretary’s  counterparts  from  the  other  overseas 
presence  agencies. 

8 

We  also  recommend  that  this  steering  committee  be 
assigned  responsibility  and  accountability  for  ensuring 
that  a  complete  and  approved  enterprise  architecture 
is  developed  and  available  to  guide  and  constrain 
overseas  presence  systems  modernization  efforts. 

9 

To  assist  the  steering  committee,  we  recommend  that 
the  Under  Secretary  direct  the  State  CIO  to  establish 
an  enterprise  architecture  project  office,  headed  by  the 
recently  named  Chief  Architect,  to  develop  and 
maintain  the  enterprise  architecture  and  submit 
appropriate  versions  of  this  architecture  to  the  steering 
committee  for  approval. 
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DESCRIPTION 

10 

We  further  recommend  that  the  State  CIO,  at  a 
minimum,  ensure  that  the  project  office  (1)  inciude 
staff  from  each  of  the  overseas  presence  agencies,  (2) 
integrate  the  architecture  with  these  agencies' 
respective  operationai  and  systems  environments,  and 
(3)  adhere,  as  appropriate,  with  the  federal  CIO 

Council"  published  guidance  on  managing  enterprise 
architectures. 

11 

To  further  address  State’s  need  to  implement 
modernization  management  controls,  we  recommend 
that  the  Under  Secretary  for  Management  direct  the 
State  CIO  to  ensure  that  the  overseas  presence 
system  modernization  office,  consistent  with  State's 
project  management  requirements  and  recognized 
models  for  effective  software  acquisition,  establishes 
and  implements  rigorous  and  disciplined  acquisition 
processes,  including  processes  for  acquisition 
planning,  contractor  solicitation,  requirements 
development  and  management,  contractor  tracking 
and  oversight,  test  and  evaluation,  transition  to  support 
the  acquired  system,  and  risk  management. 

12 

Until  these  recommended  modernization  controls  are 
in  place,  we  also  recommend  that  the  Secretary  limit 
further  investment  in  the  overseas  presence 
knowledge  management  modernization  program  to  (1) 
conducting  ongoing  prototyping  and  limited  pilot 
testing  and  (2)  establishing  needed  modernization 
management  controls. 
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GAO  Comments 


1.  See  comments  2  through  12. 

2.  We  did  not  address  the  adequacy  of  funding  for  near-term  activities  as 
part  of  this  review. 

3.  We  disagree.  Our  report  clearly  links  our  findings,  conclusions,  and 
recommendations.  Specifically,  the  report  states  that  the  level  of 
management  rigor  and  discipline  needed  to  effectively  manage  prototype 
and  pilot-testing  activities  is  not  as  demanding  as  that  needed  for 
acquisition  and  deployment  of  a  system  that  is  to  be  used  operationally. 
For  example,  while  State’s  lack  of  an  enterprise  architecture  to  guide  and 
constrain  prototype  and  pilot-testing  activities  does  not  introduce 
significant  risk,  it  would  introduce  considerable  risk  for  State  to  begin  the 
next  phase  of  the  program  without  this  architecture.  Accordingly,  we 
concluded  that  State’s  existing  management  controls  were  not  adequate 
for  the  next  phase  of  the  program,  and  we  made  recommendations  that  are 
intended  to  proactively  provide  a  framework  for  effectively  managing  the 
more  demanding  next  phase  of  the  program.  We  made  no 
recommendations  intended  to  correct  problems  with  State’s  management 
of  its  near-term  activities  because  we  concluded  that  controls  over  the  less 
demanding  prototype  and  pilot-testing  activities  were  adequate. 

4.  We  disagree.  Our  report  clearly  describes  how  State’s  existing 
management  controls  are  not  adequate  for  the  next  phase  of  the  program. 
For  example.  State  does  not  yet  have  an  enterprise  architecture  to 
effectively  guide  and  constrain  investment  in  the  system.  Our  experience 
with  federal  agencies  has  shown  that  attempting  to  define  and  build  major 
systems  without  first  completing  and  using  an  enterprise  architecture 
often  results  in  systems  that  are  duplicative,  not  well  integrated, 
unnecessarily  costly  to  maintain  and  interface,  and  not  effectively 
optimizing  mission  performance.  Similarly,  State  has  not  yet  implemented 
an  effective  process  for  managing  requirements  for  an  operational  system, 
as  evidenced  in  the  report  by  the  fact  that  agency  partners  have  not 
formally  approved  requirements  for  the  prototype  and  pilot  systems. 
Although  not  having  formally  approved  requirements  for  building 
prototypes  and  conducting  pilot  tests  does  not  introduce  significant  risks, 
it  would  introduce  considerable  risk  for  State  to  acquire  a  system  that  is  to 
be  operationally  deployed  without  having  formally  approved  requirements. 

5.  We  agree  that  State  has  not  proposed  program  and  management  plans 
beyond  the  prototype  and  pilot-testing  activities. 


Page  44 


GAO-02-41  State-Led  Modernization  Initiative 


Appendix  II:  Comments  From  the  Department 
of  State 


6.  We  agree.  State’s  plans  provide  for  preparing  a  prototype  and  pilot 
testing  evaluation  report.  Also,  such  a  report  will  be  needed  by  not  only 
the  lead  agency  for  acquiring  and  deploying  the  operational  system,  but 
also  all  participating  agencies  that  will  need  to  integrate  the  operational 
system  with  their  respective  legacy  systems. 

7.  See  comments  3,  4,  and  5. 

8.  We  agree.  As  stated  in  our  report,  the  Secretary  of  State  was  directed  to 
lead  a  cabinet-level  committee  to  implement  the  recommendations  of  the 
Overseas  Presence  Advisory  Panel,  which  included  acquiring  and 
deploying  a  multiagency  common  platform  knowledge  management 
system.  While  we  recognize  the  possibility  that  State  might  not  be  the  lead 
agency  for  the  next  phase  of  the  program,  we  strongly  believe  that,  to 
allow  for  program  continuity  and  avoid  unnecessary  delays,  these  issues 
must  be  addressed  now.  Since  State  is  currently  the  lead  agency  for  the 
program,  the  recommendations  are  being  directed  to  the  Secretary  of 
State. 

9.  We  agree  with  State’s  comment,  and  note  in  our  report,  that  the  roles, 
responsibilities,  and  accountabilities  of  State  and  its  agency  partners 
would  need  to  be  fully  and  formally  defined  before  moving  beyond 
planned  near-term  activities.  Although  State  asserted  during  the  course  of 
our  review,  and  in  its  comments,  that  its  present  informal  structure  for 
managing  agency  relationships  is  working  and  its  comments  assert  that  it 
provided  documents  to  us  attesting  to  this  on  numerous  occasions,  this 
documentation,  while  voluminous  and  repetitive,  did  not  demonstrate 
what  the  current  process  is  and  whether  it  was  working.  As  a  result,  we 
were  forced  to  expand  the  scope  of  work  to  include  a  survey  of  State’s 
agency  partners  to  determine  whether  State  had  engaged  them  in  program- 
related  activities. 

From  our  survey,  we  were  able  to  conclude  that  State  had  taken  adequate 
action  to  engage  the  agencies.  Nevertheless,  some  agencies  raised 
concerns  that  must  be  addressed  before  beginning  the  next  phase  of  the 
program.  For  example,  one  agency  partner  stated  that  it  had  not 
participated  in  an  approval  process  for  program  planning  documentation, 
another  partner  stated  that  it  was  unaware  of  any  approval  process  for 
planning-related  activities,  and  a  third  stated  that  it  had  not  reviewed  any 
of  the  program  planning  documentation.  Moreover,  one  agency  partner 
stated  that  a  more  formal  program  governance  structure  is  needed, 
especially  for  reviewing  and  approving  such  documents  as  the  program’s 
concept  of  operations  and  the  system’s  functional  requirements.  A  similar 
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statement  was  made  in  a  meeting  of  the  State-led  Foreign  Affairs  Systems 
Integration  Architectural  Working  Group.  Specifically,  it  was  stated  that 
protocols  needed  to  be  established,  such  as  officially  listing  the  documents 
that  would  be  shared  so  that  all  participants  would  be  aware  of  these 
documents  and  have  the  opportunity  to  review  them  before  making  any 
decisions  about  how  to  proceed.  While  these  issues  are  less  significant  for 
program  near-term  activities,  they  should  be  resolved  before  moving 
beyond  system  prototyping  and  piloting  activities.  Accordingly,  our  report 
contains  proactive  recommendations  for  doing  so. 

10.  We  agree  with  State’s  comment  that  the  need  to  formally  define  agency 
roles,  responsibilities,  and  accountabilities  should  be  focused  on  the  next 
phase  of  the  program.  We  disagree  that  there  is  a  need  to  eliminate  a 
reference  to  insufficiency  in  defining  existing  roles,  responsibilities,  and 
accountabilities  regarding  near-term  program  prototype  and  pilot-testing 
activities  because  no  such  reference  exists.  In  fact,  our  report  concludes, 
and  State  acknowledges  in  its  comments,  that  management  controls  over 
near-term  activities  are  adequate. 

11.  See  comments  3  and  4. 

12.  See  comments  3,  4,  and  5. 
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